an entry from

Piotr's R&D blog

Early this year James Chisan, myself, and a gang of undergrads (Jeff Cockburn, Reid Garner, Azarin Jazayeri and Jesse Wesson) developed an application I whimsically called the Video Bench. Against expectations, the application has taken on a life of its own and we recently demoed it at CASCON, getting a very warm reception from the audience. This entry explains what the application is about, what I learned from presenting it at CASCON, and what the future might hold for the Video Bench.

Genesis

It all started when Peggy obtained a Diamond Touch tabletop from MERL and was looking for some enterprising students to take it through its paces. Having recently seen Minority Report, I came up with the idea of using the table for gesture-driven video editing. To keep the project grounded, I decided to take the lead from old cut & paste film editing techniques, where strips of film were physically cut with a knife and pasted back together with tape. The design was fully sketched within days, and the Video Bench implemented in under 6 weeks.

The Video Bench is a hands-on collaborative video editing application meant for casual users. To operate it, you sit on a special conductive mat in front of the Diamond Touch surface and use your hands to manipulate strips of video. Multiple people can use the app simultaneously without interfering with each other. The operations permitted on video strips are mostly pretty simple: play, pause, fast forward & rewind, cut & paste, copy & trash, zoom, and spread.

This last operation requires a bit of explanation. Since there's clearly not enough space on the tabletop to show every frame of each strip, the frames are collapsed into "cels". At first, each video clip loaded into the Video Bench is represented by a strip with only one cel. By moving back and forth through the video, the user can locate the desired point and cut the strip in two. However, navigating through video in this manner is tiresome (if familiar); we can do better. We allow the user to spread video by grabbing the edges of two cels in a strip and pulling them apart. As the edges get further apart, the space is filled with more cels that further subdivide the video between these two points. This is a kind of timeline zoom, where the time axis is partially projected onto the X axis, and is one of the few really novel ideas in this project.

For more details on the Video Bench, and copious amounts of screenshots and diagrams, please refer to our group's final report, keeping in mind that a few things have changed since it was written.

The unveiling

Once the report was written and submitted, I thought the project was pretty much over. However, Peggy suggested that we show it off at CASCON. James and I agreed, and we were soon booked into a prime spot of real-estate on the show floor. I'll spare you the troubled tale of troubles we went through to get all the equipment flown across the country and set up in its new location; suffice it to say, we were ready on D-day.

The exhibit was immensely popular with the public; we had a crowd of people around our booth whenever it was staffed. Most of that can be attributed to the inherent flashiness of the application in the middle of a rather cookie-cutter technology showcase (how exciting can you make a computer and monitor look, anyway, no matter what it's displaying?), but some people were genuinely interested in the concept and had some good suggestions. I also managed to surrepetitiously observe one person attempting to operate the Video Bench when the booth was unattended, which produced further insights on the user interface.

The most striking impression was that people were taking the Video Bench seriously. We had one person ask when (and how) we were planning to commercialize the prototype, and a number of people expressed interest in using the app. We knew, by design, that this app would only appeal to casual users, who are not willing to learn complex user interfaces. I was surprised, however, when a semi-professional videographer claimed that some clients are put off just looking at a professional video editing tool, even if they don't have to use it. Using the Video Bench to rough out a production would result in a much less threatening environment.

I also got some ideas about how to improve the user interface. The jogging operations are nearly useless: they are too imprecise to achieve frame-perfect positioning, yet too slow to quickly scan through video. Instead, dragging the cursor directly in the strip's top edge for absolute positioning, and having a relative positioning control in the bottom edge would be a better combo. (The relative positioning control would fast forward or rewind at a speed proportional to the finger's distance from some center point.) Another issue was moving strips around: people don't expect to be able to use full-hand gestures, and even once instructed they can be difficult to get right consistently. With the jog gestures out, we could use a single-finger drag in the cel to move strips. Alternatively, the adventurous user discovered that the dividers were "live", and tried to use them to move strips around. He was very confused when he discovered that the strips only moved horizontally (remember that the dividers are only meant for spread/fold). It might be a good idea to also use dividers as movement handles, as this would also allow for natural strip rotation when using two fingers!

Finally, other people mentioned that there's video-related work going on at the NRC and at UofT. (For DT-related work, see the next blog entry on spacewarps.) Another person thought that this kind of collaborative surface would be great for bioinformatics work; I didn't catch the details, but it involved visualizing protein structures and manipulating them collaboratively. Other people were keen on seeing this used for software engineering. When I brought up the fact that it's difficult to create content with fingers, they suggested that participants could have personal tablets for input to the communal surface.

Overall, people's enthusiasm and the many ideas I gathered make me think this project has some life in it yet.

A nebulous future

What's next for the Video Bench? We'll probably move it to the next generation prototype of the DT, and exhibit it a few more times at local venues. The previous section gave a few ideas for user interface refinements, and my next blog entry explores other perspectives. Brian Corrie at NewMIC did some work on a distributed version of the app, and has some other improvements in the pipeline. Ultimately, though, the Video Bench is looking for a new home: this is not mine (or James') primary area of research, and sooner or later (probably sooner) we'll have to focus on our dissertations. If the Video Bench is to survive and prosper, the project needs new blood. Anyone?