It’s been a long time since I touched Livid’s VJ software. But when it comes to hardware, they’ve been doing unbelievable work that could have Resolume and VDMX and Processing and vvvv and Max users turning their heads.
The Ohm64 looks simply fantastic. It’s got an ideal configuration for a lot of live visual applications, with 8×8 triggers and plenty of faders and knobs to go along with them. It’s also finally a controller that’s well-made but doesn’t cost a fortune – you get a well-crafted device made by the people who designed it in Texas, but at $599, it’s still affordable.
Visualists are doing all kinds of new things to expand their performance, so I believe having a truly open controller is essential. The Ohm64 delivers, with a chip and editor software that have extensive open support. That means that, as with the brilliant monome controller, you should see a community that experiments with creative ideas for how to use it. (Nor do I think this is necessarily monome competition – the monome is still beautiful for its minimalism, whereas this should appeal to people who ignored the monome because they needed knobs and faders for additional parameter control.)
For visual software increasingly using OSC, a future firmware update should provide native OSC support (and possibly even DMX in the near future). For everything else, there’s MIDI support now. And unlike the Akai APC40, that means real MIDI support, with actual MIDI in and out ports and endlessly customizable controller assignments and LED feedback, instead of the Akai’s single USB port and permanently-fixed layout. And this is fully bus-powered, so you’re not screwed if you forget your power brick.
I did a full preview for CDMusic. But next week I should get to try one in person, which is the real test.
Kamel Makhloufi posts this appetite-whetting image of the “augmented reality” ARToolKit working on the iPhone platform. Sure, Microsoft and Sony may be hard at work on computer vision applications that don’t require tags, but for mobile applications, tags seem perfect. This is just a preview, but it already suggests some interesting art applications, as well as the usual games and such. I’ve been impressed with how well the camera on my Android-powered TMobile G1 works, even in low light, so the fact that tags are designed for easy readability by digital cameras suggests this could be eminently practical.
I just hope we also see ARToolKit on Android, too, naturally.
That was fast – Kyle McDonald notes in comments that some folks have already baked some parallel tracking and mapping system work on the iPhone, for markerless tracking. This sort of confirms for me why I like markers – there’s just so much data in this image to deal with, and presumably you have more you want to do in an app than just take care of tracking.
Having flowers pop out of this does look beautiful, and there is something magical about watching this happen without the markers.
By the way, since we’re looking at yet another iPhone video, I’m not advocating working on Android out of some sort of sense of ethical purity. Put simply, I just personally see the long-range development experience and deployment as being more flexible and broader. And I don’t think that thinking “mobile” has to be about any one platform. Ideally, you’d come up with solutions that allows you to get your artwork or game or whatever it is on iPhones and Androids and other devices.
Artist Lia has created her first piece of art for the iPhone and iPod touch, something called PhiLia 01. It’s a quirky, gorgeous generative sound and visual app activated by movement, one that encourages users to save their own artwork.
Composer Morton Subotnik used to talk really eloquently back in the days of multimedia CD-ROMs and The Voyager Company about computers as “chamber music” environments. Instead of seeing the personal scale of technology as an impediment, he viewed it as something intimate and wonderful. So it’s fantastic to see artists engage mobile platforms as a way to have that relationship with a participant.
There’s now also a page up that is beginning to collect some of these particular artworks, focusing on generative-style interactive creations, and featuring work by our friend Memo Akten. Joshua Davis’ kaleidoscopic artmaking tool Reflect, which he showed for the first time at OFFF earlier this month in Lisbon, is enroute.
The way in which these tools are being created is interesting, too. PhiLia is built in OpenFrameworks, the open source C++-based development tool made friendlier for artists with integrated toolsets, a community of friendly creative folk, and simplified, speedy syntax similar to Processing. OpenFrameworks, thanks to its open source nature, has made its way onto the iPhone.
Part of what this demonstrates is that, while the iPhone itself is proprietary, some of the power of open source can still triumph. And, indeed, by basing work on this open source foundation, these same artists aren’t imprisoned by a single platform. PhiLia could be a desktop app, or on other mobile platforms once they support OpenFrameworks.
And, yes, it means I’m aching that much harder to get OpenFrameworks and/or Processing onto Android – it should be possible. (Java on Android is not identical to Java on desktop, so it can’t be a direct port – you can’t just install Processing on Android – but it is possible.) There are also still some wrinkles in the App Store approval process; it really is refreshing on Android (and presumably things like Palm WebOS) not to have those restrictions.
Then again, that’s the whole point: OS and specific platform shouldn’t have to matter, and open source software – and artwork – can be just as brilliant on a proprietary platform as an open one.
You can thank Lee Byron, Memo Akten, Damian Stewart, Zach Gage, and the core OF team (Zach, Theo, and Arturo). The “power of open source” is not some sort of magical whirlwind that surrounds code and makes things appear spontaneously – it’s blood, sweat, and tears (unpaid!) by real people. Although, if you get those real people together in a room and do some sort of battle shout or Care Bear Stare (sorry, I’m an 80s kid), it might help psych you up.
Microsoft’s Project Natal unveiling for Xbox 360 was no question a blockbuster of technology presentations, nothing short of sheer magic in a games industry that has lately looked somewhat backward-looking. The combination of a 3D-capable camera with facial and object recognition and vocal recognition and mic interaction takes already-smart elements and puts them together into something bigger. But demos are just that – it’s the reality of what’s happening in interaction design that’s interesting.
So, some more details on Project Natal:
Latency?
Note that the video in the post yesterday carries a significant disclaimer: it’s essentially a conceptual mockup, not a real demo. In videos we’ve seen of the current prototype, there does seem to be a significant lag between an action and its representation on the screen. This may have to do with the sheer amount of data and analysis that’s being done on it. Unfortunately, as this is only in prototype stage, it’s impossible to do much more than speculate.
I’m not the only one to notice this: Keith Lang, interaction designer at Plasq, sees the same concern in his (excellent) round-up of coverage of Project Natal:
Don’t underestimate how important the latency could be, either. Even tiny differences in latency can have a major impact on how someone feels about an interaction. This is also significant to music people, who generally like their interactions to use tiny latencies and approximate the rate of the audio they’re controlling.
I’ll reserve judgment until the final version, naturally! But it’s something to watch.
Johnny Chung Lee and the 3D Technology
The ingenious creator of various Wii tracking hacks, it seems, is now with Microsoft. (Nintendo, your loss. Rest of the world, he has code tools on his site, so even without hiring the guy, you can benefit from his knowledge.) Cristian Campo spots the news in our comments.
For his part, Johnny is careful to note that he’s not responsible for what you see, but is working with them on productization.
He can’t reveal anything but what’s public, but he does have some more extensive details on the technique – essentially, information that is public but in a more technically-specific form:
The 3D sensor itself is a pretty incredible piece of equipment providing detailed 3D information about the environment similar to very expensive laser range finding systems but at a tiny fraction of the cost. Depth cameras provide you with a point cloud of the surface of objects that is fairly insensitive to various lighting conditions allowing you to do things that are simply impossible with a normal camera. But once you have the 3D information, you then have to interpret that cloud of points as "people". This is where the researcher jaws stay dropped. The human tracking algorithms that the teams have developed are well ahead of the state of the art in computer vision research. The sophistication and performance of the algorithms rival or exceed anything that I’ve seen in academic research, never mind a consumer product. At times, working on this project has felt like a miniature “Manhattan project” with developers and researchers from around the world to coming together to make this happen.
We would all love to one day have our own personal holodeck. This is a pretty measurable step in that direction.
Creepy Kids
Seaman, you’ve got nothing on this. (Sorry, Leonard Nimoy.)
Yes, it seems Peter Molyneux’s latest project uses Project Natal to simulate interactions with a kid. This does start to make me wonder if – as “realityengager” wonders in CDM comments – we should just go out into the real world and interact with that. (Daddy? Why won’t you play with me any more? Why are you only playing with Xbox 360 Milo kid?) But as a tech demo, of course, it’s mind-boggling – and it’s nice to think what it might mean for storytelling.
See the video at top. Molyneux suggesting that even science fiction hasn’t written about this sort of technology is especially absurd, as it seems science fiction spends most of its time writing about exactly this, but you get the point.
I just want Project Natal support in XNA so artists can play with this stuff. Hear us, Microsoft?
Anyone for a game of Harmonix Mime Hero, with the Marcel Marceau expansion pack?
We’ve seen simple computer vision applications, “augmented reality” systems and object tracking schemes that use specially-printed tags, 3D tracking using IR emitters, and specialized motion detection sensors (most notably Nintendo’s Wii). But the holy grail, of course, is getting tracking without any of that stuff. That’s the idea behind the widely-anticipated release today of Microsoft’s Project Natal for Xbox 360.
What’s different about the new tracking systems that makes them work better? In short, a z axis. By detecting depth from the camera, you can track motion in three dimensions, which in turn makes detecting specific gestures far easier.
Microsoft had acquired 3D motion detection system maker 3DV Systems, as confirmed earlier this year on VentureBeat. Today’s news: that technology will see commercial distribution. Project Natal for Xbox 360 uses a three-camera device that interprets z-axis depth. Already, this leads to some impressive game demos. Of course, a big challenge of the Nintendo Wii has been that its sensors work poorly, but another challenge has been that developers often don’t use the sensors well, either. So it remains to be seen if developers figure out just what to do with this stuff.
There’s more, too:
3D motion detection and tracking
Facial recognition (which could in turn lead to multi-person control experiences with this sort of technology, because you can tell the difference between different people)
“Object scanning” – no mention of object detection, but this could mean tangible interfaces that don’t require special tags
5
