I’ve been blogging effectively every day for nearly five years, so it’s hard to avoid novelty - that’s kind of what we publish. The question is, is it the novelty that’s important, or do you see these as little steps toward something greater that hasn’t happened yet? I tend to favor the latter.
Here’s the question: can augmented reality - using computer vision tools to mix computer graphics with stuff in the real world - become a real medium and not just a gimmick?
If it works in a game, you begin to think there’s really something to an interactive design. I’m not certain yet about the upcoming PSP game from giant Sony. Georgia Tech researchers, on the other hand, are coming up with stuff I really want to play.
Oh, yeah - and it makes me crave some Skittles. (Product integration, anyone?)
Apparati Effimeri write to let us know about their latest visualist projection mapping project, “TETRAGRAM FOR ENLARGEMENT.” Watching abstract patterns wend their way across the geometries of the castle-like building is hypnotic, to be sure. But it’s as the visuals make the building seem to melt, collapse, and reconstruct itself into fluid shapes worthy of Dali that things
get really interesting.
I have to ask: is it really club culture that has “failed” the visualist, or just the limited architectures of the clubs we have now? Projection mapping has tended to be the domain of “art” visuals. But I wonder if music and visuals alike might benefit from sharing new spaces. Heck, I’d be happy to go dance outdoors to adventurous music programming while visuals crept around the buildings around me.
So, do spread this stuff around, because sometimes the live event metaphor in which a lot of us find ourselves is something very different — the silo.
You’ve seen the demos. You like the idea of tracking tags in the real world to create visuals. And now you want to try augmented reality for yourself - and, incidentally, you’re a Flash developer.
Reader Eric Socolofsky writes to share a framework he’s created that makes it much easier to work with the Flash-based, open source FLARToolkit, called FLARManager. Version 0.4 is just released:
FLARManager has a number of features that improve upon the existing work done by FLARToolkit:
Building the apps themselves is easier. Fire up the framework with Flex Builder (or Flash, or Eclipse, or FlashDevelop), and you have access to all the libraries you need, so you can start playing more or less out of the box. Hello, world, indeed.
You don’t have to rely on Papervision if you don’t want to. Papervision, the faux-3D library for Flash, is included with the distribution. But marker tracking is decoupled from Papervision, so you don’t have to use it if you don’t need it.
Better event management. Marker adding, updating, and removal, multiple pattern detection and management, and the like are all extended in FLARManager.
Great documentation. Eric has taken the time to read some fantastic getting started tutorials, all accessible from the site above so you can go play.
Now, you wouldn’t pick Flash for speed - that’s not the idea.
Imagine what’s now possible with a mobile phone: anyone with a supported device can jam with other artists, walk up to an installation, connect to other creators and other software, all using supported protocols. Leaving behind the days of painstaking manual adjustment of MIDI commands and obscure drivers, and even the act of having to physically connect gear, software - and with it, digital art - can simply talk to each other in standard ways.
That’s why we’re excited about software like Memo Atken’s MSA Remote. It uses the standardization provided by the network-savvy, open protocol OSC, with additional plug-and-play (or, erm, don’t plug, do play) functionality from the TUIO protocol. OSC provides the communication; TUIO makes the messages standardized.
To avoid confusion: You do NOT need a Mac to use OSC. OSCulator is a cool app - and makes bridging to MIDI easier - but it’s just one tool among many. You can use this app with Windows and Linux, too, and visual apps like VDMX, Resolume Avenue, Pd/GEM, Processing… the list goes on. In fact, almost every visual app today worth using uses OSC, even as the music world is painfully slow to catch on.
Technology still has the power to appear like magic. And one place we may desperately need magic: straightening out our horribly shaky, handheld video shots. Software makers like Apple have already offered up some techniques for doing this - in the case of Apple’s Final Cut Studio, optical flow analysis attempts to track the image as it shakes around the screen and compensates by adjusting the orientation of the frame. But a research team at the University of Wisconsin, partnering with Adobe, will present a new approach at the legendary graphics-geeky SIGGRAPH conference in August. They go one step further, applying a 3D mesh to the image to warp your image three-dimensionally to make the stabilization even more seamless.
Me writing about it is basically useless. Check out the mind-blowing results in the video. From the description:
In this paper, we describe a technique that transforms a video from a hand-held video camera so that it appears as if it were taken with a directed camera motion. Our method can adjust the video to appear as if it were taken from nearby viewpoints, allowing for 3D camera movements to be simulated. By aiming only for perceptual plausibility, rather than accurate reconstruction, we are able to develop algorithms that can effectively recreate dynamic scenes from a single source video. Our technique first recovers the original 3D camera motion and a sparse set of 3D, static scene points using an off-the-shelf structure-from-motion system. Then, a desired camera path is computed either automatically (e.g., by fitting a linear or quadratic path) or interactively. Finally, our technique performs a least-squares optimization that computes a spatially-varying warp from each input video frame into an output frame. The warp is computed to both follow the sparse displacements suggested by the recovered 3D structure, and avoid deforming the content in the video frame. Our experiments on stabilizing challenging videos of dynamic scenes demonstrate the effectiveness of our technique.
You can view all the techie details there, as well as many more demo videos. This is promising stuff, and we’ve seen in recent years a vast acceleration of the time between academic research and shipping commercial products — especially with cheap computational power on home computers to play around with, and increasing challenges for software vendors to differentiate what they’re doing in a mature application space.
Side note: boy, do I want to go to SIGGRAPH this year.
1
