Kamel Makhloufi posts this appetite-whetting image of the “augmented reality” ARToolKit working on the iPhone platform. Sure, Microsoft and Sony may be hard at work on computer vision applications that don’t require tags, but for mobile applications, tags seem perfect. This is just a preview, but it already suggests some interesting art applications, as well as the usual games and such. I’ve been impressed with how well the camera on my Android-powered TMobile G1 works, even in low light, so the fact that tags are designed for easy readability by digital cameras suggests this could be eminently practical.

I just hope we also see ARToolKit on Android, too, naturally.

Via Flickr.

That was fast – Kyle McDonald notes in comments that some folks have already baked some parallel tracking and mapping system work on the iPhone, for markerless tracking. This sort of confirms for me why I like markers – there’s just so much data in this image to deal with, and presumably you have more you want to do in an app than just take care of tracking.

Having flowers pop out of this does look beautiful, and there is something magical about watching this happen without the markers.

By the way, since we’re looking at yet another iPhone video, I’m not advocating working on Android out of some sort of sense of ethical purity. Put simply, I just personally see the long-range development experience and deployment as being more flexible and broader. And I don’t think that thinking “mobile” has to be about any one platform. Ideally, you’d come up with solutions that allows you to get your artwork or game or whatever it is on iPhones and Androids and other devices.

Anyone for a game of Harmonix Mime Hero, with the Marcel Marceau expansion pack?

We’ve seen simple computer vision applications, “augmented reality” systems and object tracking schemes that use specially-printed tags, 3D tracking using IR emitters, and specialized motion detection sensors (most notably Nintendo’s Wii). But the holy grail, of course, is getting tracking without any of that stuff. That’s the idea behind the widely-anticipated release today of Microsoft’s Project Natal for Xbox 360.

What’s different about the new tracking systems that makes them work better? In short, a z axis. By detecting depth from the camera, you can track motion in three dimensions, which in turn makes detecting specific gestures far easier.

Microsoft had acquired 3D motion detection system maker 3DV Systems, as confirmed earlier this year on VentureBeat. Today’s news: that technology will see commercial distribution. Project Natal for Xbox 360 uses a three-camera device that interprets z-axis depth. Already, this leads to some impressive game demos. Of course, a big challenge of the Nintendo Wii has been that its sensors work poorly, but another challenge has been that developers often don’t use the sensors well, either. So it remains to be seen if developers figure out just what to do with this stuff.

There’s more, too:

• 3D motion detection and tracking
• Facial recognition (which could in turn lead to multi-person control experiences with this sort of technology, because you can tell the difference between different people)
• “Object scanning” – no mention of object detection, but this could mean tangible interfaces that don’t require special tags

Magic is itself a kind of augmented reality, a willing suspension of disbelief as we watch what we know is a blend between what we’re seeing and what we simply think we’re seeing. We know it’s not all physically happening, but the act of seeing it is enough. So it’s fitting that someone would try to blend magic and augmented reality. The challenge is flirting with a taboo of the digital age: getting people to accept that digital magic can still be magical, and not just empty illusion.

Marco Tempest is a rare character who combines magic and technology. He’s an unabashed showman in the old mold: you know it’s a show, and you know he’s selling what he’s doing. He also has some terrific ideas, and he knows his tech. Marco’s sends us his latest augmented reality, which he says is entirely real-time.

And here comes the reversal of the usual magic trick. In the pre-digital age, you’d keep the illusion alive by sharing as little as possible. Explain the tools, and the trick may be ruined. But in the post-modern, post-digital world of sharing and open source, it’s the reverse. You actually need to share something in order to have credibility. (My guess is that this could be the trend for far more than just magic – look at the musician and visualist worlds.)

Marco writes:

this is 100% real-time stuff - No post-processing. Programmed In C++ with OpenFrameworks, OpenCV, ARToolkitPlus, MacCam and other Open Source goodies…

Being the fan of software pr0n that I am, I thoroughly enjoy the screenshots, which show off the OpenFrameWorks setup [site | cdmo tag]. Incidentally, you can use OpenCV not only with C++, but Java/Processing, as well – check out our tutorial. And Bryan Chung has been working on a version of the ARToolkit library for Processing, too. That’s not to take away from OFW – it’s a really powerful environment, and there remain advantages on the C side.

Here’s what it looks like behind the magic:

 More screenshots

The pixel-mangling madness continues. Here are the latest goodies to show up in our inbox.

Enrico Nau aka naus3ayt is playing with code-for-artists environment Processing (site | cdmo tag). It’s yet another example that reveals that computer vision, aside from the useful-ish applications like tracking what you’re doing, can be used to purely aesthetic ends. No code excerpt here, but the best place to begin here would be the official Processing tutorials or the OpenCV tutorial recently posted here on CDM.

Blog post here:

Chroma-sampler: painting with light

Enrico describes his creation to us:

I recently coded a toy that samples colors from a video feed and used them to create virtual brushes; similar mood to data moshing, different approach.

In other news, we have this gorgeous glitchy … something or other. Okay, actually, I have no idea what this is or where it came from (and neither does tipster con), but boy, does it take the opposite tack compared to the one suggested by David O’Reilly – it’s a maximalist, intentional overuse of a particular data compression artifact. In this case, though, it fits nicely with the musical content, which is what O’Reilly suggested.

godxilliary.com [QuickTime link]

I guess my ban on datamosh, incidentally, is well over, especially after I temporarily moshed the story database on Create Digital Music by importing with some setting wrong. (It doesn’t help that WordPress and MySQL once defaulted to Swedish. Swedish: the new lingua franca.)

Anyway, enjoy – and keep experimenting, everyone. Experiments, by definition, can be ugly, and that’s good. It’s all part of the process.

Closing out our (unexpected) Week of Optical Flow feature, Michel from the vvvv forums has ported Andrew Benson’s optical flow implementation to that DirectX-based, Windows-only, free for non-commercial-use patching environment. (Jitter and Quartz Composer, mentioned earlier, each use OpenGL, not DirectX.)

Since it is DirectX, the shader uses HLSL instead of GLSL. In technical terms, folks, that means it’s one letter more different. I can illustrate: Gary the gregarious gorilla would become Harry the hreharious horilla. You see? (Kidding. HLSL is actually closer to NVIDIA’s Cg, but all these things are built on the same basic principles - and they’re all just shader languages.)

This is just a first go at this patch, so vvvv users wanting to improve upon it, go for it!

Distortion From Optical Flow incl. Optical Flow Shader