Technology still has the power to appear like magic. And one place we may desperately need magic: straightening out our horribly shaky, handheld video shots. Software makers like Apple have already offered up some techniques for doing this - in the case of Apple’s Final Cut Studio, optical flow analysis attempts to track the image as it shakes around the screen and compensates by adjusting the orientation of the frame. But a research team at the University of Wisconsin, partnering with Adobe, will present a new approach at the legendary graphics-geeky SIGGRAPH conference in August. They go one step further, applying a 3D mesh to the image to warp your image three-dimensionally to make the stabilization even more seamless.

Me writing about it is basically useless. Check out the mind-blowing results in the video. From the description:

In this paper, we describe a technique that transforms a video from a hand-held video camera so that it appears as if it were taken with a directed camera motion. Our method can adjust the video to appear as if it were taken from nearby viewpoints, allowing for 3D camera movements to be simulated. By aiming only for perceptual plausibility, rather than accurate reconstruction, we are able to develop algorithms that can effectively recreate dynamic scenes from a single source video. Our technique first recovers the original 3D camera motion and a sparse set of 3D, static scene points using an off-the-shelf structure-from-motion system. Then, a desired camera path is computed either automatically (e.g., by fitting a linear or quadratic path) or interactively. Finally, our technique performs a least-squares optimization that computes a spatially-varying warp from each input video frame into an output frame. The warp is computed to both follow the sparse displacements suggested by the recovered 3D structure, and avoid deforming the content in the video frame. Our experiments on stabilizing challenging videos of dynamic scenes demonstrate the effectiveness of our technique.

The research, at the University of Wisconsin-Madison:
Content-Preserving Warps for 3D Video Stabilization

You can view all the techie details there, as well as many more demo videos. This is promising stuff, and we’ve seen in recent years a vast acceleration of the time between academic research and shipping commercial products — especially with cheap computational power on home computers to play around with, and increasing challenges for software vendors to differentiate what they’re doing in a mature application space.

Side note: boy, do I want to go to SIGGRAPH this year.

Also along these lines: Spacetime Fusion, tests of Final Cut’s SmootCam feature, more SmoothCam tests

For those of you purists, yes, it’s still worth considering the art of steadicam shots - at least before technology obliterates it for us clueless masses. Previously: B&H Interviews Steadicam Inventor: Shooting is Like Dancing

While we’re on the subject of User Interface…

Adobe, as makers of the dominant creative software that powers the planet, are certainly due some criticism. But beware the wrath of the angry user. Via lili katschen on Twitter, I see the Intersphere has its own forums for endless Adobe gripes.

Now, don’t get me wrong: I have Adobe critiques of my own. Those have more to do, however, with the dangers of monolithic, commercial tools, and the missed creative opportunities they can engender. I believe in the power of digital technology, so I think the alternative can be not only to pick up a paintbrush or a Bolex camera, but to try a simpler tool or DIY creation. No matter how good Adobe’s software is, if you’re trapped inside it, it’s no longer useful. Escape it, and you can come back to a CS4 with fresh eyes.

But these sites read more like an endless bug log. No app - even a simple one - would stand up to that scrutiny. And, ironically, you begin to see that the problems often lie not with Adobe but the development frameworks on which they have to rely. (Even that may be a good argument for more open source development, not exclusively, but as a way to begin to build better frameworks for the future.)

Anyway, that’s not why I bring you this. I actually wish Adobe apps would misbehave more, because these glitches start to look like wonderful abstract art.

Dear Adobe, a relatively friendly site that works like a Twitter for bug reports and does get some Adobe responses, starts to read like haiku:

“should a gaussian blur really bring my system to a stall?”

“Quality is better than quantity”

“i love you but you’re expensive” (hmmm, that could apply to so much…)

“I miss HomeSite.”

I can almost hear Creative Suite, the opera. Philip Glass-like piano ostinatos play in the background as designers with funky glasses and just-unbuttoned shirts step in and out of spotlights, murmuring the final echoes of a digital graphic design age…

Adobe UI Gripes is more of an ongoing, out of control juvenile tirade. Oddly, though, its contributors seem to have hit upon some fantastic mode in which Adobe CS becomes a glitch workstation.

You know what this means: someone needs a Glitch Suite GS4. vade, can you get on this?

And yes, I think it’s time to stop thinking about user interface altogether and begin imagining what our computers could do if these crusty 80s UI relics could melt off the screens for a day.

While not intending it, the blogs have given us a glimpse of what that process might be like, in a post-apocalyptic GUI wasteland.

All images via Adobe Gripes. Seriously, guys, keep these coming — they’re gorgeous. A lot better than some of the actual work people create with CS.

Artist Patrick Gunderson creates spectacular canvases of particle-generated paintings, a bit like code action paintings. (Action as in the likes of Jackson Pollack, not just ActionScript.) Above, an image from the Create Digital Motion Flickr pool. It’s worth having a closer look on Flickr - as with a physical painting, the closer you get, the more detail you see.

This is a Actionscript 3 generated composition. Color samples are taken from a base image then drawn using a psudo-random line drawing algorithm in concert with a particle system.

The technique itself is not exclusive to Mr. Gunderson, though I really love his use of color in the composition. Now, all due respect to ActionScript, I just hope he takes a look at some Processing and OpenGL code, as this could be even more fun set in motion, for doing some things Flash may not be quick enough to do. (As you can tell from the name of the site, we have a certain bias toward things that move ’round these parts.)

Really beautiful work, and further evidence that the aesthetics of code can go in all sorts of directions. Definitely check out the many variations in Patrick’s Flickr set; there’s quite a lot more than can fit here.

Updated: I wrote this post quickly and really should have mentioned the work of Erik Natzke, whose techniques Patrick adapted and who is the major artist in this field. Erik’s stuff really deserves its own post, which I didn’t have time to do justice here, so — will remedy that by tomorrow.

Software is increasingly a medium for artists. That to me raises really deep questions about platforms, and whether you’ll have freedom with the platforms you use. A gig’s a gig, and there’s nothing wrong with using these tools to accomplish jobs. But on the other hand, for the same practical (not even philosophical) reasons, the ability to modify the platform when you push past its capabilities is essential.

The ability to take your creations and run them on a variety of mobile devices - and how much freedom you have with the tools on which they’re built — is a key issue in 2009, as mobile platforms coalesce.

As I see people making amazing artwork with code, and then we have this question of which of our semi-disposable devices will actually be able to run stuff, I think this is as much an artistic question as a technological question. So it’s worth evaluating how three big players are doing at the moment. And this isn’t just to advocate open source for the sake of it: a big question is whether you’ll be able to write code and get it on mobile screens around the world.

Adobe has quietly added some features to Premiere CS4 I would consider really huge. Adobe Updater is normally the little app that delivers Acrobat Reader updates what seems like every other weekday, but imagine my surprise when I opened it today: a little New Year’s present.

• AAF support (Advanced Authoring Format, an attempt to create a more intelligent interchange format that’s open to developers, backed by Adobe)
• OMF export (Open Media Framework, as used by various apps including Cubase, SONAR, Pro Tools, and notably as an alternative in Final Cut … not to be confused with the fantastic early 90s One Must Fall PC game … heheh)
• Final Cut Pro import (yes, that’s native, not via OMF)

And this isn’t just about wooing current Final Cut users: fully supporting OMF and AAF is an essential prerequisite to making media production more open and standards-based. Apple, of course, already helped create a world that isn’t strictly Avid-based, which is why they’ve also been backers of some of these initiatives.

Nothing against Final Cut, necessarily, but having alternatives is good – particularly when Apple’s flagship doesn’t even run on all Macs (sorry, non-Pro MacBook). Now, I had largely looked to Sony Vegas in the past, but Adobe has been busily updating Premiere and so far, CS4 looks far, far snappier performance-wise. Dealing with these file types is tricky, but if Adobe actually has nailed some of the import/export options, it’ll be very good news. And, anyway, if it works for you in your editing workflow, that’s really the main test, isn’t it?