You’ve seen the demos. You like the idea of tracking tags in the real world to create visuals. And now you want to try augmented reality for yourself - and, incidentally, you’re a Flash developer.

Reader Eric Socolofsky writes to share a framework he’s created that makes it much easier to work with the Flash-based, open source FLARToolkit, called FLARManager. Version 0.4 is just released:

http://words.transmote.com/wp/20090618/flarmanager-v04/

FLARManager has a number of features that improve upon the existing work done by FLARToolkit:

• Building the apps themselves is easier. Fire up the framework with Flex Builder (or Flash, or Eclipse, or FlashDevelop), and you have access to all the libraries you need, so you can start playing more or less out of the box. Hello, world, indeed.
• You don’t have to rely on Papervision if you don’t want to. Papervision, the faux-3D library for Flash, is included with the distribution. But marker tracking is decoupled from Papervision, so you don’t have to use it if you don’t need it.
• Better event management. Marker adding, updating, and removal, multiple pattern detection and management, and the like are all extended in FLARManager.
• Great documentation. Eric has taken the time to read some fantastic getting started tutorials, all accessible from the site above so you can go play.

Now, you wouldn’t pick Flash for speed - that’s not the idea.

Fans of vintage games with Macs, take note. Open Emu makes emulation of classic game systems a “first-class citizen” on the Mac. But if it were just a game emulator, well, it wouldn’t be news. What makes it news is that at its core, Open Emu is an open source platform and modular architecture into which your favorite game systems can be added as plug-ins. And thanks to that architecture, you can treat your favorite game systems as though they’re modules in a grand, 8-bit modular visual synth, crunching their textures into geometry, adding real-time effects, and controlling the whole thing with multiple controllers, audio, and MIDI.

In other words, Open Emu is like having a giant visual performance synth made from the tasty innards of classic games.

The platform has been in feverish development for some time, but today a major new release takes it further. Beta 2 of 1.0 extends the modularity of the platform, adds a finished Quartz Composer interface (allowing integration with other apps, live visuals, and graphical, modular patching using Apple’s development tool), and adds more emulation cores.

Supported game systems for emulation: Sega Master System, Game Gear, SG-1000, Game Boy, Game Boy Color, Game Boy Advance, NES/Famicom, SNES/Super Famicom, Sega Genesis, Sega CD, and Sega 32x. (In the immortal words of Strongbad, ain’t got no Turbografx? Sounds like you just need one more plug-in.)

Game emulation is nothing new, but the programming team has build a friendly Mac front-end for a host of mature, popular emulation engines. They also fully support Mac technologies, even third-party niceties like Sparkle for automatic updates.

But, this being Create Digital Motion, we’re interested in the live visualist-friendly features:

• High-quality OpenGL scaling, multithreaded playback, and other optimizations
• Audio or MIDI actually plays the game (and can also be used to make the game line up with music)
• Play multiple ROMs at the same time
• Real-time 3D effects, image processing - and route game controllers to those effects

In other words, you can jack in your favorite MIDI controller and go nuts with your favorite games, turning them into a live performance medium - then mashing up the resulting textures with real-time, 3D/2D effects. The Nestopia engine supports ROM glitching, cheat codes, and game rewinding — essential so that in-game death doesn’t also kill your set, and so you can play with the aesthetic of glitchy cartridges without blowing on a classic game cart.

ToonLoop Remix from Society for Arts and Technology on Vimeo.

Yes, you read that right: realtime stop motion. While stop motion is, by definition, associated with a painstaking process of creating animation frame by frame, a free and open source tool takes a different approach. ToonLoop provides the usual stop motion tools for creating loops, but takes a live performance approach to the recording and playback process, so you can turn your stop motion into a performance. The creator brought up the tool Saturday at the Open Video Conference in New York and got just the reaction you’d expect - a few confused (if delighted) chuckles, and someone asking, “That must be … slow.”

Now, if the framerate is low, you have no one to blame but yourself.

For fans of animation and live visualism, though, this is a dream. The first build was in Processing for Mac and Windows, but a new version for Linux (which should also work on Mac) is built on Python (with PyOpenGL, PyGame, Video4Linux and — oddly — Pure Data for MIDI).

http://toonloop.com/
Developers: Alexandre Quessy and Tristan Matthews
Toonloop Download
Source on Google Code
More documentation of the project at Montreal’s SAT [in French]

In fact, I’m not sure whether I should tell you to download the thing or just run with the idea itself. (There’s no reason Java/Processing shouldn’t still work, by the way, if you use the excellent GSVideo library - and OpenFrameworks and others could be likely candidates, too.)

The idea is brilliant - and yet more evidence that being a visualist can be a much broader category than simply being a “VJ,” with the two-channel mix paradigm the more conventional term suggests.

And performances evidently look like what you might expect. Below, Joy Penroz uses Toonloop in Mérida, Yucatán, México, via the ToonLoop site.

Bonus video: as I was looking for more work done with ToonLoop (there’s not much out there just yet), I came across another creation by Joy Penroz. It’s not a stop motion performance, but it runs with parallel ideas, looping to manipulate time in a modern pop take on the work of Dutch master painter Jan Vermeer. The contemporary “Milkmaid”:

THEMILKMAID from Joy Penroz on Vimeo.

Extra thanks to Michela Ledwidge and Austin Gambles on Twitter.

Puddle of Life - Darwin Exhibit from Tiago Serra on Vimeo.

Here’s a really elegantly-designed multi-touch table. It uses two really key pieces of open source technology: the Community Core Vision (“CCV”) tracker, formerly known as tbeta, and a lovely framework for coding visuals called PyMT.

PyMT, as the name implies, is a Python-based framework. I’ve gotten to know Nathanaël Lécaudé, a talented artist and coder who was nice enough to put me up a couple of nights while I was in Montreal; he’s one of several core coders. They’re doing a lot to really encapsulate functionality in widgets in a nice way. Features of PyMT include an event framework, specialized widgets for gesture, touch, and layout, and connections to OpenGL, OpenGL shaders, and sound. You can even work with the enduring, evergreen synthesis language Csound using its Python bridge, the oddly-named but powerful Ounk.

Of course, that’s all plumbing. It’s nice to see this applied to something powerful and educational. From the Vimeo description:

"Puddle of Life" is an educational multi-touch installation that was designed for Coimbra’s Science Museum (Darwin exhibit), demonstrating the theory of natural selection, as part of Darwin’s 200th anniversary.

The installation is composed of a round multi-touch table whose surface represents a virtual environment where 4 species of creatures live in. Each of these little creatures have different physical characteristics visually expressed by it’s quantity and/or size: Vision, Locomotion and Fur.

The player interacts with the game by touching the surface. He has to maintain full awareness of his creatures emotions (visually represented by a cartoon like ballon) and using this information to properly choose the mating partners on the control console. This console also allows the user to select the most appropriate descendent from 4 possible mutations, resulting from the reproduction.

The player’s objective is to help the species he controls achieve the highest number of creatures of its kind. Since this world suffers from climate change the player must assure that his creatures are well adapted to this ever-changing environment by choosing the best balance between mating partners and descendants.

The species move in a swarm, but they split when the predator is near. When a creature sees food it warns the nearby siblings and they all run towards it. Of course only the ones who have best locomotion reach it. The vision gives them the ability to see the predator sooner and the ability to see the food further ahead too. The fur is useful to them when the temperature is low but harmful when it’s hot, leading the creature to a shorter lifespan.

The player wins when they reach about 18 living creatures and loses when all of it’s creatures die from cold/hot, predation, famine or old age.

Photos: flickr.com/photos/tserra/sets/72157619458503007/with/3619335559/

Technical Specs:
LLP multi-touch table
CCV tracker - nuicode.com/projects/tbeta
pyMT framework - code.google.com/p/pymt/
Cython - cython.org/
Rabbyt sprite library for pyglet - matthewmarshall.org/projects/rabbyt/
Animation and artwork: Adobe’s AE, Ai and PS with Fasticon.com icons

The project is the work of Tiago Serra and his collaborators, who have also been PyMT contributors (see comments). Great work, gang. We’ll be watching for more.

We have the tools. We have the techniques. Now, what happens when you put technology for tracking physical objects into the hands of artists around the world?

On June 6, members of the CDM community joined in our first “global hackday,” assembling tangible interfaces on tracking tables. Martin Kaltenbrunner of reacTIVision and the reacTable joined us from Vienna, Austria, while Adam Kumpf of the OpenFrameworks-powered Trackmate and MIT Tangible Media Group chatted from Cambridge, Massachusetts.

Much of the day was about saying “hello, world,” and helping each other through getting cameras working, troubleshooting, and the like. But there was an extraordinary variety of ideas even in one day. I can only imagine where things might go from here. I can also see the tools people were developing as being expressive for live visuals and visual performance - and via a really cheap interface, too.

Some of the accomplishments of our first day:

• Endless ideas: Drawing interfaces using objects, a floor tom as a housing, a musical instrument with soda bottles as the interface, a game with blocks featuring the Tokyo skyline, and others.
• Troubleshooting data: Both the Trackmate and reacTIVision projects got lots of feedback about how people were using the tools, where tracking was and wasn’t working, and where people got stuck up. We also compiled lots of information on cameras, drivers, builds, and operating systems. I’m working with Adam and Martin to dig through a lot of this information so we can compile a really practical guide to make it easier for people to create their own projects.
• Special guests: Marco had his augmented magic show and we had the beginnings of an interactive glove. Check out the video highlights to see what the NYC event was like.
• noisepages for networking, and other tools: Livestream video was a bit of a mixed bag; I’m still looking for easier ways of doing that (both on the video shooting side and the streaming side.) Text chat was an easy win, though IRC can still be cumbersome; I’m looking into integrating standardized XMPP chat instead, and providing access via any client or a webpage. But the other big success story was that noisepages worked nicely for documentation; see the fritzcrate and i3games sites for great examples!

Building Communities Around the World

A real highlight to me was getting to hear from Valeria (jalea.tv) and Jose (Estado Lateral Media Lab), visiting New York from Argentina. They talked about what the scene is like in Buenos Aires, and touched on issues of community, learning, open source, and the multilingual world of coding. And they do some really beautiful and hip visual work, both commercial and experimental. We also wound up with a significant amount of the online chat being in Spanish. That to me is a healthy sign - “global” really doesn’t necessarily translate to “English.”

Hackday Results, and the Future

For more documentation, head to our noisepages site:

Tangible Interface Hackday: The Projects (So Far)
http://hackday.noisepages.com/

http://trackmate.sourceforge.net/
http://reactivision.sourceforge.net/

And, yes, this is only the beginning. My suspicion is that a single weekend would be enough to get workable tracking projects going - especially as we iron out some technical wrinkles. But we’d certainly love to do more of this, whether it’s another “official” hackday or simple an open lab with chat and sharing.

To continue this moving forward, you can join the Tangible / Multi-touch Interfaces group on noisepages.com:
Tangible + Multi-Touch noisepages Group

And I’d like to brainstorm how we might proceed, whether it’s a formal event or a sort of online open lab. You can join that conversation on the group:
Let’s make every day a Global Hackday - the event continues

noisepages is still in alpha state, but we’re working aggressively to move forward to beta, and content placed there is safe and future-proof. (Most importantly, I’ve fixed the jpeg library on the server so that avatars work!)

We look forward to hearing from you.