Microsoft’s Project Natal unveiling for Xbox 360 was no question a blockbuster of technology presentations, nothing short of sheer magic in a games industry that has lately looked somewhat backward-looking. The combination of a 3D-capable camera with facial and object recognition and vocal recognition and mic interaction takes already-smart elements and puts them together into something bigger. But demos are just that – it’s the reality of what’s happening in interaction design that’s interesting.

So, some more details on Project Natal:

Latency?

Note that the video in the post yesterday carries a significant disclaimer: it’s essentially a conceptual mockup, not a real demo. In videos we’ve seen of the current prototype, there does seem to be a significant lag between an action and its representation on the screen. This may have to do with the sheer amount of data and analysis that’s being done on it. Unfortunately, as this is only in prototype stage, it’s impossible to do much more than speculate.

I’m not the only one to notice this: Keith Lang, interaction designer at Plasq, sees the same concern in his (excellent) round-up of coverage of Project Natal:

Microsoft Announces ‘Natal’ 3D System [UI&us]

Don’t underestimate how important the latency could be, either. Even tiny differences in latency can have a major impact on how someone feels about an interaction. This is also significant to music people, who generally like their interactions to use tiny latencies and approximate the rate of the audio they’re controlling.

I’ll reserve judgment until the final version, naturally! But it’s something to watch.

Johnny Chung Lee and the 3D Technology

The ingenious creator of various Wii tracking hacks, it seems, is now with Microsoft. (Nintendo, your loss. Rest of the world, he has code tools on his site, so even without hiring the guy, you can benefit from his knowledge.) Cristian Campo spots the news in our comments.

For his part, Johnny is careful to note that he’s not responsible for what you see, but is working with them on productization.

Project Natal [procrastineering]

He can’t reveal anything but what’s public, but he does have some more extensive details on the technique – essentially, information that is public but in a more technically-specific form:

The 3D sensor itself is a pretty incredible piece of equipment providing detailed 3D information about the environment similar to very expensive laser range finding systems but at a tiny fraction of the cost. Depth cameras provide you with a point cloud of the surface of objects that is fairly insensitive to various lighting conditions allowing you to do things that are simply impossible with a normal camera.

But once you have the 3D information, you then have to interpret that cloud of points as "people". This is where the researcher jaws stay dropped. The human tracking algorithms that the teams have developed are well ahead of the state of the art in computer vision research. The sophistication and performance of the algorithms rival or exceed anything that I’ve seen in academic research, never mind a consumer product. At times, working on this project has felt like a miniature “Manhattan project” with developers and researchers from around the world to coming together to make this happen.

We would all love to one day have our own personal holodeck. This is a pretty measurable step in that direction.

Creepy Kids

Seaman, you’ve got nothing on this. (Sorry, Leonard Nimoy.)

Yes, it seems Peter Molyneux’s latest project uses Project Natal to simulate interactions with a kid. This does start to make me wonder if – as “realityengager” wonders in CDM comments – we should just go out into the real world and interact with that. (Daddy? Why won’t you play with me any more? Why are you only playing with Xbox 360 Milo kid?) But as a tech demo, of course, it’s mind-boggling – and it’s nice to think what it might mean for storytelling.

See the video at top. Molyneux suggesting that even science fiction hasn’t written about this sort of technology is especially absurd, as it seems science fiction spends most of its time writing about exactly this, but you get the point.

I just want Project Natal support in XNA so artists can play with this stuff. Hear us, Microsoft?

Microsoft Research have added live, collaborative visuals to their bag of tricks for the Microsoft Surface multitouch table. Of course, in the process, they’re really demonstrating not only what you might to with Surface but with multitouch interfaces in general. In fact, it’s particularly odd that Microsoft hasn’t apparently made the connection with more generally-available multitouch hardware coming out, particularly with multitouch APIs built directly into Windows 7. HP is already shipping a mainstream laptop with a convertible, tablet-like form factor. And I don’t need to point out that this could lead to cross-platform, open source applications, not just those that run at Microsoft tech demos or on a unit installed in Vegas.

Via Ars Electronica:

VPlay: live video mixing meets Microsoft Surface (Subhead: “It’s like VJing on a Microsoft Surface!” Uh… actually, more than just like that, I’d say it is that.)

Thanks to Pedro Marques (VJ Danger) for the tip!

The ideas here, if in basic prototype form, are already interesting. The design is heavily influenced by (if not a direct copy of) the Reactable, down to the connectors between nodes.

http://mtg.upf.es/reactable/

Visuals arguably work even better, though, because they can be shown directly in a way sound cannot.

In this prototype, you can’t do much that you can’t already do with conventional visual software. But already, there are two significant, fundamental advantages. One is, having a nodal visual interface gives you really open-ended possibilities for setting up a set. Conventional software also relies on you to configure the modules you need in advance of performing. With this interface, you do it all live as you go – just as you can do with sound on the Reactable. Secondly, as the video points out, you can collaborate more easily, without fighting over knobs on your MIDI controller.

I couldn’t help but laugh at this particular frame from the video (which also, amusingly, heavily features Resolume v2):

Yeah? Speak for yourself. Us VJs are fighting the boys and girls off. We’re like fresh meat in a shark tank.

This solution is much better. Now people can get beer all over the inside of my multitouch controller. Give me the solitary existence any day.

Hi from Multitouch Barcelona on Vimeo.

For those of you who think that there’s a guy living inside your computer that makes all the magic happen… now there can be.

The gang at Multitouch Barcelona (recently spotted at OFFF) have posted a cheeky project that imagines a real human interface. Microsoft Bob has nothing on this.

I love the way in which this pulls apart the notions behind all these interfaces, and especially the use of the space inside the box. So, could we see practical applications? Will we all be communicating soon by remote multi-touch? (I’ll leave you to imagine the naughty or downright absurd and slapstick implications of that.)

Via preciousforever / Christophe Stoll on Twitter.

D-func, one third of German DVJ trio Weissgold.TV, picked up a Lemur [on CDMo, on CDMu] a couple of weeks ago, and has put together a fantastic custom interface which controls VDMX [on CDMo] over two machines.

VIDVOX Forums - I finally found my perfect set-up!

VDMX’s UI customization is one of its biggest strengths, allowing you to create the setup which is just right for the job you’re doing. Combining this with the Lemur - a controller whose interface you create to fit the job you want it to do - gives a very sleek, unique setup. Weissgold are projecting on a pyramid, and have a tab of the interface devoted to setting this up:

NAB, the “broadcasting” industry show, remains the event of the year for visual gear lust fantasies. We kicked off this week with new gear from Edirol, but that was just the beginning. Contributor Anton Marini (”vade”) weeds through the rest of the announcements for us, and finds some very tasty-looking equipment if you’re interested in getting HD video into your computer in real-time, or recording HD-resolution computer performances. The combination of this hardware with our faster-than-ever computers means that HD VJing and visualism is now more accessible than ever. -Ed.

NAB ‘09 is winding down, and there have been a slew of announcements of new products and upgrades that run the gamut of super high-end real-time 4K playback systems to.. well, not so high-end. I’ve tried to pick through the details and find the announcements that may help change the game for visualists in 2009/2010, for both high-end professional VJs and hobbyists alike.

The key word this year is HDMI.

AJA Ki-Pro

The Aja Ki-Pro is a field recorder. That’s a fancy way of saying it’s a stand-alone device that records audio and video to a hard drive. While Ki Pro is aimed more at higher-end production and post-production markets, it does allow visualists with the budget to do one thing we’ve all been wanting: Record your performances in HD, without compromise.