You’ve seen the demos. You like the idea of tracking tags in the real world to create visuals. And now you want to try augmented reality for yourself - and, incidentally, you’re a Flash developer.

Reader Eric Socolofsky writes to share a framework he’s created that makes it much easier to work with the Flash-based, open source FLARToolkit, called FLARManager. Version 0.4 is just released:

http://words.transmote.com/wp/20090618/flarmanager-v04/

FLARManager has a number of features that improve upon the existing work done by FLARToolkit:

• Building the apps themselves is easier. Fire up the framework with Flex Builder (or Flash, or Eclipse, or FlashDevelop), and you have access to all the libraries you need, so you can start playing more or less out of the box. Hello, world, indeed.
• You don’t have to rely on Papervision if you don’t want to. Papervision, the faux-3D library for Flash, is included with the distribution. But marker tracking is decoupled from Papervision, so you don’t have to use it if you don’t need it.
• Better event management. Marker adding, updating, and removal, multiple pattern detection and management, and the like are all extended in FLARManager.
• Great documentation. Eric has taken the time to read some fantastic getting started tutorials, all accessible from the site above so you can go play.

Now, you wouldn’t pick Flash for speed - that’s not the idea.

What happens when digital cinematography meets a favorite desktop video editing app for mortals? When our friend Nathanaël Lécaudé, also a talented multitouch developer, said he had encountered some work with RED, I was really curious to know first-hand what the experience was like. Sony Vegas is a curious creature - it’s the name you hear least when talking about desktop video editing, until you talk to users, at which point this Windows-only tool gets near-cult status. It’s especially big among visualists because its original developers built it from a musical perspective. Throw in the RED camera’s digital cinematography, though, and something interesting happens. -PK

While I’m not rich enough (yet ;)) to own a RED camera, I’ve been pretty curious to see what the editing workflow is like with different apps.

When Sony Vegas 9 came out, the first feature that got my attention was RED support. I have always loved Vegas for its simplicity and excellent workflow, so I wanted to learn if it would still be beautiful with the huge RED files. I downloaded a couple of R3D files from http://www.redrelay.net. Here are a few observations:

• You can open Vegas and just drag the r3d files, they will scale to the project resolution you are in. No need to convert the files or use proxies.
• If you want full 4k support, you need to change your project settings for 4k (there’s already a preset for that).
• I was able to achieve 12fps playback on my Core 2 Quad CPU, which seems pretty good. [Ed.: For the file size, I'd say that's really terrific - and Nat's PC wasn't an expensive machine, either.]
• You can set your project resolution lower (to 720p) and realtime playback will bump to full framerate (24fps).
• Audio will be imported as 4 separate tracks (this is great as Vegas’ audio tools are very good).
• At 4k, you can do a lot of cropping before losing quality, which is very nice.

What was really impressive for me was the ability to change the RED raw settings directly from Vegas. Among other things, you can set white balance, exposure, custom curves, color space, all within Vegas. You can see the result in realtime. I do a lot of RAW photo editing
with tools like Adobe Lightroom, and being able to do the same with video files is magical.

I tried the same thing in Premiere Pro CS4 but wasn’t able to get the files on the timeline (Premiere would say the files were not of a recognized format). Now, I’m not a Premiere expert, so it might be my fault. [Ed. Still, the ability to get Vegas' speed and workflow here to me is a huge boost - and the fact that Vegas, even with RED, still can run happily on modest machines. -PK]

There’s a very good article about all of this here :
Going one better: RED editing and what Vegas 9 has under the hood [Broadcast Newsroom]

Ed.: I hope Nat forgives me for doing this, but I have to post this shot of him with the RED camera. This thing is just huge. It looks, at least, like a friendly robot. The idea that anyone would go out and buy a car when they could have one of these instead just baffles me. -PK

2009 will be remembered as the E3 game event that embraced computer vision. Far from me-too answers to the Wii’s gestural controllers, we saw remarkably different visions of how computer tracking might work.

As expected, Sony had their own motion tracking system to unveil at their press conference. But unlike Microsoft’s 3D camera, Sony opted to build on their already-lovable PlayStation 3 Eye camera with wands with spheres. The controllers look ridiculous, and lack the magic of the Microsoft demos. But don’t dismiss them out of hand. (Sorry, there’s no way to write this story without lots of abstract puns.)

Much of what Microsoft showed was “conceptual” video – and some of the hands-on demonstrations had noticeable latency problems. Sony’s approach, meanwhile, was really quite literal in its demonstation. The tracking looks extremely accurate in 3D space, and latency appears to be minimal.

Above: Video of the press conference – check out how quick and accurate the tracking looks

Via Joystiq; see also Offworld’s excellent 5 Things You Need to Know about the Sony shindig

The other good news for people working as artists and not necessarily mass-market game developers is that you can start to play with these ideas right now. Whereas Microsoft seems to have “lost” the once publicly-available 3D camera SDK for their solution, Sony is using an off-the-shelf camera you can buy right now and doing the rest of the work in software. I really like the use of tangible interfaces with cameras, because you can get more predictable tracking results, and you get the tactile feedback of having something in your hands. (I’m not sure I’d be as excited as they are about having a glowing ball on the end, but maybe I need to channel my inner raver.)

Anyway, here’s my humble prediction: it doesn’t matter how cool the demo looks or what sweeping statements anyone makes. Gameplay alone matters, and that means that what has to happen next is dependent entirely on the tracking working reliably and quickly, and developers building smart stuff around it that works as games. The same, naturally, is true for anyone doing broader interaction design and live visuals.

Sony is also getting further into the augmented reality arena. They have a Tamigotchi/Nintendogs-style augmented reality pet simulator, EyePet, for the console (see Joystiq’s hands-on), plus Invizimals, an augmented reality title for the PS3. Of the two, Invizimals is the most interesting. It’s funny that they immediately design it for kids (too bad, as I can see some office antics with this sort of thing). It’s also evident just how hard designing an effective augmented reality game can be. I don’t think skepticism would be wildly out of place – it’s clear that there’s something powerful about the concept, but not clear just what it will be.

And I don’t need to remind you, if you haven’t joined our tangible interface virtual party Saturday, head to http://hackday.noisepages.com/ – ARToolkit augmented reality is very much on the plate of stuff we’d like to see people play with. (The other schemes we’re using, Trackmate and reacTIVision, are better suited to 2D tracking on a surface, though they’re very, very reliable for that task.)

For one of the final Game On performances, Melbourne’s “spiky electro pop” group Flamingo Crash came along and rocked out in front of a bunch of security cameras, and alongside their visual collaborator Simulcast with his Tagtool.

They enjoyed the live footage so much that they commissioned me to edit it as an “offical” music video for their album.

Flamingo Crash - Sister Sister from Jaymis on Vimeo.

I’m seeing this kind of visualist- and web-savvy behavior coming from bands with increasing regularity. Artists (and audiences) are starting to realise that a music video doesn’t need to be blinged out and post-produced into sterility to be entertaining and valid. Outfits such as La Blogotheque’s “Take Away Shows” and $99 music videos are showcasing consistently high quality releases of low-budget, high-speed concepts. As a rule, visualists - those who can perform, produce, hack, and create new concepts and looks quickly - are going to do well in this environment. In the financially exciting world we have right now. Focussing on “fast and effective” also means that you’re able to release more work, which in turn attracts more people to your work, which allows you to release more work… Personally, I’ve more offers so far in 2009 than in all of 2008, so I don’t see this process reversing itself any time soon.

Nine Inch Nails have been on tour around Australia with the Soundwave festival, and their artistic director Rob Sheridan has been along for the ride, and took his Canon 5D MKII on stage at various venues.

Full HD Version from Nine Inch Nails.

Screw resolution charts and slow focus pulls on flora, this is how you test out a camera. Take it up on stage with crushing noise, blistering lights, smoke, sweat, and guitarists flailing around. The 5D has taken it all in stride. All of the videos are available for download in 1280×720. That’s a 650MB file for the above video.

Obviously Sheridan’s close collaboration with Trent and NIN has allowed him to work in a way which many artists would find intrusive. Weaving amongst the performers, and handling the manual focus with aplomb, he’s pulled off the most realistic, personal concert footage I’ve seen for a long time.

Update: Here’s a crowd view of the same performance, including Rob moving around the stage. Interesting to remind yourself how a “normal” camera deals with concert lighting.