Fans of vintage games with Macs, take note. Open Emu makes emulation of classic game systems a “first-class citizen” on the Mac. But if it were just a game emulator, well, it wouldn’t be news. What makes it news is that at its core, Open Emu is an open source platform and modular architecture into which your favorite game systems can be added as plug-ins. And thanks to that architecture, you can treat your favorite game systems as though they’re modules in a grand, 8-bit modular visual synth, crunching their textures into geometry, adding real-time effects, and controlling the whole thing with multiple controllers, audio, and MIDI.

In other words, Open Emu is like having a giant visual performance synth made from the tasty innards of classic games.

The platform has been in feverish development for some time, but today a major new release takes it further. Beta 2 of 1.0 extends the modularity of the platform, adds a finished Quartz Composer interface (allowing integration with other apps, live visuals, and graphical, modular patching using Apple’s development tool), and adds more emulation cores.

Supported game systems for emulation: Sega Master System, Game Gear, SG-1000, Game Boy, Game Boy Color, Game Boy Advance, NES/Famicom, SNES/Super Famicom, Sega Genesis, Sega CD, and Sega 32x. (In the immortal words of Strongbad, ain’t got no Turbografx? Sounds like you just need one more plug-in.)

Game emulation is nothing new, but the programming team has build a friendly Mac front-end for a host of mature, popular emulation engines. They also fully support Mac technologies, even third-party niceties like Sparkle for automatic updates.

But, this being Create Digital Motion, we’re interested in the live visualist-friendly features:

• High-quality OpenGL scaling, multithreaded playback, and other optimizations
• Audio or MIDI actually plays the game (and can also be used to make the game line up with music)
• Play multiple ROMs at the same time
• Real-time 3D effects, image processing - and route game controllers to those effects

In other words, you can jack in your favorite MIDI controller and go nuts with your favorite games, turning them into a live performance medium - then mashing up the resulting textures with real-time, 3D/2D effects. The Nestopia engine supports ROM glitching, cheat codes, and game rewinding — essential so that in-game death doesn’t also kill your set, and so you can play with the aesthetic of glitchy cartridges without blowing on a classic game cart.

C4D CamGripTools from Cinema 4D Tutorials on Vimeo.

Deride the mouse as you will. When it comes to two-dimensional control, the device is pretty amazing. It’s a reasonable way of amplifying small hand gestures into bigger gestures on a 2D plane. But what about 3D interfaces? Suddenly, the mouse becomes like playing charades, telling someone else what to do in a universe with an extra dimension.

What you need is interfaces that make sense in 3D. Some of these interfaces are out there; the missing link has been intelligent connections to software. To see how powerful this can be, look no further than Camera GripTools, a motion capture system for Maxon’s Cinema 4D modeling tool.

It works with a variety of devices:

• Behringer BCF2000 MIDI fader panel (already a popular controller among VJs/visualists)
• Nintendo’s Wiimote
• Joysticks
• Track IR 4 Pro is a head-mounted tracker for head movements. Pricing starts at just US$120, so this is absolutely a solution for mortals, and it appears to work really well – with 120 FPS tracking.
• Polhemus Patriot VR Tracker – I’d never seen this before, and it looks utterly brilliant. It’s a six degrees of freedom tracker / 3D digitizer, made up of a small sensor. And it’s only £ 1,985 … oh. Okay, never mind.

Camera GripTools [via Derrick Belcham of visuuals.com – check out his site for some gorgeous work]

For Cinema 4D users, this looks absolutely invaluable. It supports XPresso, CInema 4D’s fantastic modular, visual programming environment, with drag-and-drop support for objects. The free demo version is already pretty usable, with more powerful versions running EUR99-499 (though sadly you need the EUR499 version for full hardware support).

But I think the bigger message is how controllable this makes the 3D environment. This could be fantastic in an open source environment like Blender, or for live control in Blender, visual tools like VDMX, and custom creations in vvvv, Jitter, Processing, and the like. And while it may not be possible to make a system as accurate as the Patriot, looking at what they’re doing I suspect it should be possible to do a “ghetto” version on the cheap. (You know I’m all about that.)

Discuss.

2009 will be remembered as the E3 game event that embraced computer vision. Far from me-too answers to the Wii’s gestural controllers, we saw remarkably different visions of how computer tracking might work.

As expected, Sony had their own motion tracking system to unveil at their press conference. But unlike Microsoft’s 3D camera, Sony opted to build on their already-lovable PlayStation 3 Eye camera with wands with spheres. The controllers look ridiculous, and lack the magic of the Microsoft demos. But don’t dismiss them out of hand. (Sorry, there’s no way to write this story without lots of abstract puns.)

Much of what Microsoft showed was “conceptual” video – and some of the hands-on demonstrations had noticeable latency problems. Sony’s approach, meanwhile, was really quite literal in its demonstation. The tracking looks extremely accurate in 3D space, and latency appears to be minimal.

Above: Video of the press conference – check out how quick and accurate the tracking looks

Via Joystiq; see also Offworld’s excellent 5 Things You Need to Know about the Sony shindig

The other good news for people working as artists and not necessarily mass-market game developers is that you can start to play with these ideas right now. Whereas Microsoft seems to have “lost” the once publicly-available 3D camera SDK for their solution, Sony is using an off-the-shelf camera you can buy right now and doing the rest of the work in software. I really like the use of tangible interfaces with cameras, because you can get more predictable tracking results, and you get the tactile feedback of having something in your hands. (I’m not sure I’d be as excited as they are about having a glowing ball on the end, but maybe I need to channel my inner raver.)

Anyway, here’s my humble prediction: it doesn’t matter how cool the demo looks or what sweeping statements anyone makes. Gameplay alone matters, and that means that what has to happen next is dependent entirely on the tracking working reliably and quickly, and developers building smart stuff around it that works as games. The same, naturally, is true for anyone doing broader interaction design and live visuals.

Sony is also getting further into the augmented reality arena. They have a Tamigotchi/Nintendogs-style augmented reality pet simulator, EyePet, for the console (see Joystiq’s hands-on), plus Invizimals, an augmented reality title for the PS3. Of the two, Invizimals is the most interesting. It’s funny that they immediately design it for kids (too bad, as I can see some office antics with this sort of thing). It’s also evident just how hard designing an effective augmented reality game can be. I don’t think skepticism would be wildly out of place – it’s clear that there’s something powerful about the concept, but not clear just what it will be.

And I don’t need to remind you, if you haven’t joined our tangible interface virtual party Saturday, head to http://hackday.noisepages.com/ – ARToolkit augmented reality is very much on the plate of stuff we’d like to see people play with. (The other schemes we’re using, Trackmate and reacTIVision, are better suited to 2D tracking on a surface, though they’re very, very reliable for that task.)

Anyone for a game of Harmonix Mime Hero, with the Marcel Marceau expansion pack?

We’ve seen simple computer vision applications, “augmented reality” systems and object tracking schemes that use specially-printed tags, 3D tracking using IR emitters, and specialized motion detection sensors (most notably Nintendo’s Wii). But the holy grail, of course, is getting tracking without any of that stuff. That’s the idea behind the widely-anticipated release today of Microsoft’s Project Natal for Xbox 360.

What’s different about the new tracking systems that makes them work better? In short, a z axis. By detecting depth from the camera, you can track motion in three dimensions, which in turn makes detecting specific gestures far easier.

Microsoft had acquired 3D motion detection system maker 3DV Systems, as confirmed earlier this year on VentureBeat. Today’s news: that technology will see commercial distribution. Project Natal for Xbox 360 uses a three-camera device that interprets z-axis depth. Already, this leads to some impressive game demos. Of course, a big challenge of the Nintendo Wii has been that its sensors work poorly, but another challenge has been that developers often don’t use the sensors well, either. So it remains to be seen if developers figure out just what to do with this stuff.

There’s more, too:

• 3D motion detection and tracking
• Facial recognition (which could in turn lead to multi-person control experiences with this sort of technology, because you can tell the difference between different people)
• “Object scanning” – no mention of object detection, but this could mean tangible interfaces that don’t require special tags

Jitter works brilliantly when it comes to processing signal - and that means for signal-like work with video and textures, it’s fantastic, as well as the usual Max-y tasks like processing input from physical sensors and input devices and the like. But try to do a whole lot of sophisticated 3D work, and Jitter may not be the best tool. For game-style 3D graphics and interaction, you want some standardized rendering and scene graph tools to take care of the hard work, plus physics and other capabilities that bring together your 3D scene.

That’s why [myu], the Max - Unity Interoperability Toolkit, looks so appealing. It not only allows for bi-directional data integration (via TCP) of Max and the Unity game engine, but can dynamically pass textures between the two. For those of you comfortable patching, say, chains of shader processors in Jitter, that means you can very quickly add some of the tasty 3D scene powers of Unity. Put together your textures in Jitter, and, say, dynamically process input from a Wii Fit balance board, then bring the input data and textures into Unity. (Unity is a friendly, elegant game engine built in C# and Novell’s open-source Mono implementation of Microsoft’s .net. Unity had previously been Mac-only but with a major new release now runs on Mac and Windows.)

The toolkit is the result of research at Virginia Tech Interactive Sound & Intermedia Studio director Dr. Ivica Ico Bukvic.

Needless to say, this could have powerful implications for all kinds of live and interactive installation applications. And yes, it is all released under the GPL.

[myu] Max-Unity Interoperability Toolkit v.1.0 Released [Cycling '74 Forum]

More Max+Unity Game Engine Goodness, with Powerful Toolkit for Max, Jitter, Pd

Teaching Adaptive Music with Games: Unity + Max/MSP, Meet Space Invaders!

For other examples of combining Max and Unity - in this case for Max’s musical powers and Unity’s gaming prowess - see another story from today:

Teaching Adaptive Music with Games: Unity + Max/MSP, Meet Space Invaders! [Create Digital Music]

Updated: About those textures…

Ico follows up to answer our questions about how you might use textures with Jitter and the Unity Game Engine, via his [myu] toolkit: