The new, svelte-looking Kinect. It's not that it looks better, though, that matters: it's that it sees better. Courtesy Microsoft.

The new, svelte-looking Kinect. It’s not that it looks better, though, that matters: it’s that it sees better. Courtesy Microsoft.

It’s a new world for media artists, one in which we look to the latest game console news because it impacts our art-making tools.

And so it is that, along with a new Xbox, Microsoft has a new Kinect.

The new Kinect uses standard infrared tracking (ideal for in-the-dark footage and accurate tracking), but also returns RGB imagery. It’s 1080p, 30-60 fps (it seems tracking is at 30 fps and video at 60, but I’m reading conflicting reports). Hands-on reports say latency is reduced. If the finished product is consistent with rumors, that could be owing to more in-hardware tracking analysis; once you get to trying to do the analysis on the computer (or console), you encounter additional bottlenecks. Now, musical readers have much greater expectations of low latency than gamers, though, so it’ll be interesting to see this in practice.

The big news is tracking that gets closer to your body, breaking analysis into smaller bits. Wired, granted exclusive early access, goes into some detail about the way the tracking tech has changed. Instead of a straight depth map created by producing a 3D picture of two separate infrared-based camera images, the new tech uses “modulated” IR light. Given that this is new technology, I’m not yet clear on the specifics of that, and would love some reader feedback. (Ahem.)

The original sensor mapped people in a room using “structured light”: It would send out infrared light, then measure deformities in the room’s surfaces to generate a 3-D depth map. However, that depth map was lo-res to the degree that clothing and couch cushions were often indistinguishable. The new model sends out a modulated beam of infrared light, then measures the time it takes for each photon to return. It’s called time-of-flight technology, and it’s essentially like turning each pixel of the custom-designed CMOS sensor into a radar gun, which allows for unprecedented responsiveness—even in a completely dark room.

Xbox One Revealed [Wired.com]

Say what? Well, the basic idea is that, by using a modulated beam of light, you can determine the depth of an object by measuring the phase shift between the emitted and received light. In fact, this is very similar to the way a single IR or (with sound) ultrasonic sensor works, only using a pixel array instead of just one emitter. You can read a paper on the subject, or follow a forum discussion on the B3D board. (Thanks to Sam Tuke for posting this. Now – specifics, still, could be interesting.)

The upshot to all of this is better tracking:

  • More discrete people can be tracked independently, without having to add more Kinects (as some hackers did) – up to six, says Microsoft. And that includes tracking people if they cross one another – a major breakthrough.
  • It’s easier to distinguish between people and objects (like your couch).
  • Individual gestures can be tracked – facial gestures, or finger-by-finger tracking (as touted by other systems like Leap).

What’s missing so far: any word of how hackable the new system will be. The last time, it took hackers to get access to camera images and tracking data, even as Microsoft themselves lagged in providing an SDK for Windows. I’d like to see more openness this time, especially given how much of the hype about Kinect has been generated by hackers – and knowing that Microsoft would like more inventive independent game design (or even art) with their tool on the Xbox platform.

MIT Technology Review is evidently waiting, too:

What Will Hackers Do with the New Kinect?

That article is largely speculative, as is mine. But I can tell you, even if you aren’t planning to use individual finger gestures and the like, anything that provides more precise tracking or reduces latency will help applications and art, generally.

And that makes this look very good indeed. Here’s a hands-on video from The Verge, for a quickie:

WIRED goes into more detail:

We’ll be watching – particularly on the hacker info. If you know anything about the development picture or can make sense of this modulated IR business, please do get in touch!

  • Steve Belovarich

    It seems at the very least “openness” is the last thing on Microsoft’s mind. The connection seems to be proprietary for the new Kinect. Hopefully this isn’t a huge hurdle for hackers.

    • http://pkirn.com/ Peter Kirn

      Potentially a big mistake. Last time, MS took eons with their own Windows SDK.

    • http://pkirn.com/ Peter Kirn

      It was an interesting situation, too – Kinect hacks got all the press, but Microsoft got sales on Xbox, even though the two were effectively unrelated.

  • Brent Marshall

    Time of Flight (TOF) is a fairly new technology, but not so new that MS is the first to market. The Panasonic D-IMager has a nice advantage of being outdoor rated. The downside is the cost, ~$3000. TOF works by sending out a pulse of light from an IR array and counting the time it takes for the light to return to the camera. This brings to mind a few questions. How did MS manage to get the cost down to a sub $200 level? (less IR LEDs?) How will this affect the high end TOF market? Will we be able to use the Kinect with a new MS KinectSDK or something external like Omek Beckon or IISU? Will the Kinect play nice in a sunny environment (high IR)? Who made the hardware for MS? How many IR LEDs are on board the Kinect (it may be possible to ascertain this from the photos and measurements — 2×6 or 3×4 array)?

  • D

    Totally Scary! I don’t think I’d want to be in the same room as that thing, especially if it’s connected to the net.

  • n4cer

    Though the port for the Kinect on Xbox One is proprietary, this is no different from the original Kinect. The port on the 360 slim supplied both power (Kinect’s power requirements are greater than what a standard USB port can supply) and a USB 2.0 connection. For Kinect 2.0, it’s power + USB 3.0.

    There are Microsoft and third-party adapters available for splitting the power and data into an AC adapter and standard USB connector. In addition to PCs, the non-slim Xbox 360s required such adapters. The situation could be a bit different with Kinect 2.0 since the only ones sold separately from the Xbox One will be the PC version, which will ship next year. Though, I wouldn’t be surprised if adapters appeared prior to the launch of the PC version.
    http://blogs.msdn.com/b/kinectforwindows/archive/2013/05/23/the-new-generation-kinect-for-windows-sensor-is-coming-next-year.aspx

    The hardware for Kinect 2.0 was developed internally by MS.

    “The highlight of the story is the CMOS sensor, which we developed internally,” Spillinger says. “This design was done completely, 100 percent on this site. This is brand-new technology. There is discontinuity between this technology and the first Kinect; from the technology perspective that we are using for depth, for 3D measurement. So this was done here. On this one, this was a complete Microsoft custom design, where our engagement is directly with the manufacturer. It’s not with any third party. We did the work. We do the qualification of the parts. We do the validation of the parts. We have done everything on this one.”
    http://www.engadget.com/2013/05/21/building-xbox-one-an-inside-look/

    The only visible LEDs I’ve seen so far is a bank of three, arranged horizontally across the center of the device, possibly used for active IR illumination.
    http://cdn0.sbnation.com/entry_photo_images/8240803/xbox-one-kinect-theverge-9_1020_verge_super_wide.jpg

    http://cdn3.sbnation.com/entry_photo_images/8240829/xbox-one-kinect-theverge-4_1020_verge_super_wide.jpg

    I haven’t seen anything specific online that points to the specs of the CMOS they’re using yet. They may share those details at the BUILD conference in June. This abstract may be related:
    http://academic.research.microsoft.com/Publication/54428116/ultra-high-throughput-single-molecule-spectroscopy-with-a-1024-pixel-spad