So, in the previous post I laid out all my ‘pain points’ and annoyances with the existing service for cinema captions.

It was around October that I thought about making an iPhone app to solve all these problems. After all, an iPhone can do all the things that the Capti-View can. Perhaps it can even improve on the Capti-View system?

I did some research and uncovered some interesting facts:

The online subtitle community is incredible

How incredible?

One week after cinema release, there were subtitles online for the new star wars movie.

In less than half an hour after airing the final game of thrones episode. There were subtitles in english, spanish, farsi and french!

Basically, like most healthy online communities, people do it for the love of it. They do it for the “karma”. Some of the best subtitle authors enjoy a very good reputation for their work.

So, sourcing subtitles was not going to be an issue. They are already out there, hosted on quite a few large platforms.

There are existing subtitle apps

If subtitle apps already exist, should I bother developing one? Well, I downloaded one and checked it out. I worked pretty well except for one key point. Synchronisation.

Have you ever watched a movie where the audio slipped out of time with the video? It is very annoying and can be downright confusing. Having correctly synchronised subtitles is a total “must have” feature for any application providing this service.

The app that I downloaded used a slider bar. This is a terrible solution to “scrubbing” the subtitles into sync.

Representing an entire movie with 4 inches of screen is very silly. Sliding across 3 pixels is equivalent to about 2 mins of movie time. So, in its current format, I think the existing app in the App Store is pretty useless.

So what’s my idea? I have Two!

I have two different approaches that I’m going to use.

Idea 1: A better UI

One approach is to develop a UI that solves the problem of “scrubbing” or syncing a sub. This should be interesting.

I think that selecting subtitles using a scrolling table view would be a much better solution. But I will have to experiment.

Idea 2: A clever algorithm

The other idea is a bit more outlandish. Could an algorithm sync the subtitle by listening to the movie?

I’ll expand on this idea: Perhaps the application could use the iPhone microphone and “listen” to a sample of audio. There are plenty of open source algorithms that do a pretty good job at this.

The actor might say: “Lets get out of here. Its going to blow!”

The application would hear: “Lets get our beer Its going to snow”

But remember the application would be able to assign a weighting of certainty to the words that it heard. So perhaps “beer” and “snow” get replaced with wild.

Now we just have to search our subtitle file for the pattern “Lets get our *@#* Its going to *@#*”

It might take a few lines before it gets a possible match, but when it does it can time shift the subtitles.

Perhaps this process only needs to happen once? Maybe the first person who uses the app in “Spiderman 7” can rate the sync quality and send a small signature back to a server. This signature could be a pattern of the first minute of the film. Future viewers of “Spiderman 7” would use this signature to sync their subtitles before any audio is even spoken?

Anyway, there are lots of idea here. Before I could explore them, I would need to be able to program yeah? So that’s kind of how I got started down this road.

 

I’m going to tell a little story that requires a lot of back story. Ready?

I’m deaf. Not just a little bit deaf, but really deaf. I can mow the lawn without my hearing aids and enjoy complete silence.

I love seeing movies. But I can’t hear the dialog. So, I love foreign films. They have captions.

A year ago I found this thing at a cinema near me:

It is awesome! My wife and I enjoyed a movie date. Usually after a film, we spend about half an hour “debriefing”. I miss a lot of plot, so she explains it all. For this date however, we just discussed how awesome the film was! (Actually the film was not very good, it was Superman, so we spent an enjoyable hour discussing the various appalling aspects).

Excellent, empowering, equalising use of technology! How cool. What problems could it have?

Pretty soon, I didn’t want to see any movies without this awesome “CaptiView” gadget. This is where I ran into trouble.

Big Problem #1: Scheduling

If you go to the website, you discover that there are about 7 sessions per day of various movies. They show the movies for TODAY and TOMORROW. Would you like to plan a fun group movie event on the weekend, 4 days from now? Too bad. You won’t know the scheduled times until the day before.

The available movies rotate around quite frequently. So you are forced to just pick what is available. This is a real shame. I would use the service a lot more if I could plan ahead.

Big Problem #2: Availability

I already sort of covered this. But not all of the showings are available with CaptiView. There were no weekend spots for the new Star Wars film, I had to take my son on a Tuesday evening. Very late bedtime for both of us!

Big Problem #3: Reliability

On two occasions I have missed out on a captioned session. I drove my son 45 minutes away to see Star Wars on a weeknight and all the CaptiView dongles were flat : (. I was gutted.

The other disappointment happened when the session times were wrongly displayed on the website. I turned up for the film and it was in a non-CaptiView cinema. So I didn’t get to see my film with subs. : (

Big Problem #4: Discretion

When I ask for the dongle at the front desk, the attendant usually has no idea what I am talking about and they have to get the manager. I don’t think many people use them. This is embarrassing. Deaf people loathe bringing attention to their disability. I hate making a fuss, but there is little choice.

I am a confident deaf person, but when I was a teenager, I would have found this experience mortifying!

So, I plan to solve all these problems with a mobile app.

(more to come)