Technology still has the power to appear like magic. And one place we may desperately need magic: straightening out our horribly shaky, handheld video shots. Software makers like Apple have already offered up some techniques for doing this - in the case of Apple’s Final Cut Studio, optical flow analysis attempts to track the image as it shakes around the screen and compensates by adjusting the orientation of the frame. But a research team at the University of Wisconsin, partnering with Adobe, will present a new approach at the legendary graphics-geeky SIGGRAPH conference in August. They go one step further, applying a 3D mesh to the image to warp your image three-dimensionally to make the stabilization even more seamless.

Me writing about it is basically useless. Check out the mind-blowing results in the video. From the description:

In this paper, we describe a technique that transforms a video from a hand-held video camera so that it appears as if it were taken with a directed camera motion. Our method can adjust the video to appear as if it were taken from nearby viewpoints, allowing for 3D camera movements to be simulated. By aiming only for perceptual plausibility, rather than accurate reconstruction, we are able to develop algorithms that can effectively recreate dynamic scenes from a single source video. Our technique first recovers the original 3D camera motion and a sparse set of 3D, static scene points using an off-the-shelf structure-from-motion system. Then, a desired camera path is computed either automatically (e.g., by fitting a linear or quadratic path) or interactively. Finally, our technique performs a least-squares optimization that computes a spatially-varying warp from each input video frame into an output frame. The warp is computed to both follow the sparse displacements suggested by the recovered 3D structure, and avoid deforming the content in the video frame. Our experiments on stabilizing challenging videos of dynamic scenes demonstrate the effectiveness of our technique.

The research, at the University of Wisconsin-Madison:
Content-Preserving Warps for 3D Video Stabilization

You can view all the techie details there, as well as many more demo videos. This is promising stuff, and we’ve seen in recent years a vast acceleration of the time between academic research and shipping commercial products — especially with cheap computational power on home computers to play around with, and increasing challenges for software vendors to differentiate what they’re doing in a mature application space.

Side note: boy, do I want to go to SIGGRAPH this year.

Also along these lines: Spacetime Fusion, tests of Final Cut’s SmootCam feature, more SmoothCam tests

For those of you purists, yes, it’s still worth considering the art of steadicam shots - at least before technology obliterates it for us clueless masses. Previously: B&H Interviews Steadicam Inventor: Shooting is Like Dancing

I’m in the middle of editing a video that combines an artist interview with event footage. The supplied raw material is 10 minutes of interview footage and 45 minutes of the event, shot from a single camera. From that footage I was able to extract 5 minutes of usable interview, but just 40 seconds of the gig. It’s not that the gig video was badly shot, it was just homogenous. Medium-long shot of people dancing. Medium shot of the DJ. Over the shoulder shot of the DJ. Medium shot of girls dancing. Repeat.

This is sad, because a single camera and half an hour is plenty of time to capture a dynamic performance. The secret sauce? Close ups.

Segue - Reset (Live at Big Day Out) from Jaymis on Vimeo.

This is getting close to the absolute minimum possible for a cheap, fast music video.

2 guys, a camera, a strobe light, and a bottle of Jägermeister.

This contains all of the elements that make the quick, single shot video effective: It’s a unique concept, it’s fast and cheap to make, it will grab your attention and evoke a strong reaction, and it’s very personal. The video is for local Brisbane band DZ, who are grabbing some mindshare and attention despite being yet to release an album.

I spend a large portion of every day watching music videos, and this is the first one to have really captured my imagination since the beautifully animated, high budget “Wood” by McBess. To me, these wildly disparate works of art are both equally valid, and equally effective as music videos. However, the fact that The Mess Up took, conservatively, less than 0.5% of the time to create, means that the artists are free to create more work, and influence more potential fans (also check out their live video, including a fantastic cover of Justice - Phantom Pt. II).

There is still plenty of scope in our industry for detailed, careful, high-budget work, but if it doesn’t have that spark of originality, then you might as well drink a bottle of Jager and throw up on the floor.

Via Aaron Meyers, who’s getting ready for some fun projects at Eyebeam here in New York this week, anyone using a camera for capture, live video, or tracking needs to check out this copious thread on the OpenFrameworks forum:

beginners ~ Sony PS3 Eye

We already knew Sony’s US$40 PS3 Eye camera was a wonder; that’s why we strongly recommended its use in the tangible interface hackday hosted earlier this month. But while we’ve heard some good luck squeezing performance out of the thing on Windows and Linux, the Mac - while reliable - could use more options and performance. Theo Watson, one of the OpenFrameworks team, comes to the rescue with a patched version of the macam open source video driver - halfway down the page. (I hope his changes get rolled into macam?)

You’ll find lots of other tips, not only for the Mac but other stuff, as well.

We’ll keep collecting tips on this camera. Macam experiences, anyone? I’m still trying to successfully build the Linux driver; once I sort that out, I’ll share.

What happens when digital cinematography meets a favorite desktop video editing app for mortals? When our friend Nathanaël Lécaudé, also a talented multitouch developer, said he had encountered some work with RED, I was really curious to know first-hand what the experience was like. Sony Vegas is a curious creature - it’s the name you hear least when talking about desktop video editing, until you talk to users, at which point this Windows-only tool gets near-cult status. It’s especially big among visualists because its original developers built it from a musical perspective. Throw in the RED camera’s digital cinematography, though, and something interesting happens. -PK

While I’m not rich enough (yet ;)) to own a RED camera, I’ve been pretty curious to see what the editing workflow is like with different apps.

When Sony Vegas 9 came out, the first feature that got my attention was RED support. I have always loved Vegas for its simplicity and excellent workflow, so I wanted to learn if it would still be beautiful with the huge RED files. I downloaded a couple of R3D files from http://www.redrelay.net. Here are a few observations:

• You can open Vegas and just drag the r3d files, they will scale to the project resolution you are in. No need to convert the files or use proxies.
• If you want full 4k support, you need to change your project settings for 4k (there’s already a preset for that).
• I was able to achieve 12fps playback on my Core 2 Quad CPU, which seems pretty good. [Ed.: For the file size, I'd say that's really terrific - and Nat's PC wasn't an expensive machine, either.]
• You can set your project resolution lower (to 720p) and realtime playback will bump to full framerate (24fps).
• Audio will be imported as 4 separate tracks (this is great as Vegas’ audio tools are very good).
• At 4k, you can do a lot of cropping before losing quality, which is very nice.

What was really impressive for me was the ability to change the RED raw settings directly from Vegas. Among other things, you can set white balance, exposure, custom curves, color space, all within Vegas. You can see the result in realtime. I do a lot of RAW photo editing
with tools like Adobe Lightroom, and being able to do the same with video files is magical.

I tried the same thing in Premiere Pro CS4 but wasn’t able to get the files on the timeline (Premiere would say the files were not of a recognized format). Now, I’m not a Premiere expert, so it might be my fault. [Ed. Still, the ability to get Vegas' speed and workflow here to me is a huge boost - and the fact that Vegas, even with RED, still can run happily on modest machines. -PK]

There’s a very good article about all of this here :
Going one better: RED editing and what Vegas 9 has under the hood [Broadcast Newsroom]

Ed.: I hope Nat forgives me for doing this, but I have to post this shot of him with the RED camera. This thing is just huge. It looks, at least, like a friendly robot. The idea that anyone would go out and buy a car when they could have one of these instead just baffles me. -PK