A Kick. A Snare. A Hat: How To Produce and Perform an Audiovisual Club Piece

By Jaymis

Robert Heel is a visual musician who’s been creating some lovely instructional films on topics dear to the heart of a visualist.

Of special interest, A Kick A Snare A Hat. Documenting a little AV performance in Resolume Avenue.

A Kick A Snare A Hat from Robert Heel on Vimeo.

Open Emu: Free Game Emulation on Mac, Quartz Composer - Even VJ with Games

emuplugins

Favorite games from the 8-bit era and beyond, now with slick, Mac-friendly functionality wrapped around them. Here’s how it might look actually playing those games.

Fans of vintage games with Macs, take note. Open Emu makes emulation of classic game systems a “first-class citizen” on the Mac. But if it were just a game emulator, well, it wouldn’t be news. What makes it news is that at its core, Open Emu is an open source platform and modular architecture into which your favorite game systems can be added as plug-ins. And thanks to that architecture, you can treat your favorite game systems as though they’re modules in a grand, 8-bit modular visual synth, crunching their textures into geometry, adding real-time effects, and controlling the whole thing with multiple controllers, audio, and MIDI.

In other words, Open Emu is like having a giant visual performance synth made from the tasty innards of classic games.

The platform has been in feverish development for some time, but today a major new release takes it further. Beta 2 of 1.0 extends the modularity of the platform, adds a finished Quartz Composer interface (allowing integration with other apps, live visuals, and graphical, modular patching using Apple’s development tool), and adds more emulation cores.

Supported game systems for emulation: Sega Master System, Game Gear, SG-1000, Game Boy, Game Boy Color, Game Boy Advance, NES/Famicom, SNES/Super Famicom, Sega Genesis, Sega CD, and Sega 32x. (In the immortal words of Strongbad, ain’t got no Turbografx? Sounds like you just need one more plug-in.)

Game emulation is nothing new, but the programming team has build a friendly Mac front-end for a host of mature, popular emulation engines. They also fully support Mac technologies, even third-party niceties like Sparkle for automatic updates.

But, this being Create Digital Motion, we’re interested in the live visualist-friendly features:

  • High-quality OpenGL scaling, multithreaded playback, and other optimizations
  • Audio or MIDI actually plays the game (and can also be used to make the game line up with music)
  • Play multiple ROMs at the same time
  • Real-time 3D effects, image processing - and route game controllers to those effects

emucube

Where that modularity gets really interesting is in the Quartz Composer form of Open Emu. Here, you can apply textures to a cube, modify them with effects, cheat and rewind your way around the game, glitch out the cartridge — eventually make a live visual performance out of game textures with live gameplay and control input.

In other words, you can jack in your favorite MIDI controller and go nuts with your favorite games, turning them into a live performance medium - then mashing up the resulting textures with real-time, 3D/2D effects. The Nestopia engine supports ROM glitching, cheat codes, and game rewinding — essential so that in-game death doesn’t also kill your set, and so you can play with the aesthetic of glitchy cartridges without blowing on a classic game cart.

read more

A Visualist Cathedral: ANTIVJ’s Grote Kerk

“Visual label” ANTIVJ has made an artistic expertise of projection mapping, light sculpture, outdoor projection, and generally painting projections onto architecture and objects. I got to see their latest work in Montreal at MUTEK, and will have an interview with them up this week. In the meantime, here’s a really stunning work from the Netherlands, at the cathedral in Breda. With original music by Thomas Vaquié, played on organ by Gerard Maters with light design by Giacinto Caponio, Grote Kerk is a sublime modern, digital spectacle, a light and sound show in a tradition extending back to the magic lantern shows. All of this is performed live using software from arKaos. (Thanks to arKaos’ own Marc Nostromo, who incidentally has a fantastic blog on noisepages.)

http://antivj.com/

For another work from this crew, here’s Joanie Lemercier’s Live Painting: Shackleton, performed live in Croft, Bristol, UK at the end of last year:

AntiVJ - Live Painting: Shackleton from AntiVJ on Vimeo.

Stop Motion as Performance? Toon Loop, Free Realtime Tool; Plus a Modern Milkmaid

ToonLoop Remix from Society for Arts and Technology on Vimeo.

Yes, you read that right: realtime stop motion. While stop motion is, by definition, associated with a painstaking process of creating animation frame by frame, a free and open source tool takes a different approach. ToonLoop provides the usual stop motion tools for creating loops, but takes a live performance approach to the recording and playback process, so you can turn your stop motion into a performance. The creator brought up the tool Saturday at the Open Video Conference in New York and got just the reaction you’d expect - a few confused (if delighted) chuckles, and someone asking, “That must be … slow.”

Now, if the framerate is low, you have no one to blame but yourself.

For fans of animation and live visualism, though, this is a dream. The first build was in Processing for Mac and Windows, but a new version for Linux (which should also work on Mac) is built on Python (with PyOpenGL, PyGame, Video4Linux and — oddly — Pure Data for MIDI).

http://toonloop.com/
Developers: Alexandre Quessy and Tristan Matthews
Toonloop Download
Source on Google Code
More documentation of the project at Montreal’s SAT [in French]

In fact, I’m not sure whether I should tell you to download the thing or just run with the idea itself. (There’s no reason Java/Processing shouldn’t still work, by the way, if you use the excellent GSVideo library - and OpenFrameworks and others could be likely candidates, too.)

The idea is brilliant - and yet more evidence that being a visualist can be a much broader category than simply being a “VJ,” with the two-channel mix paradigm the more conventional term suggests.

And performances evidently look like what you might expect. Below, Joy Penroz uses Toonloop in Mérida, Yucatán, México, via the ToonLoop site.

Joy Penroz performing with ToonLoop

Bonus video: as I was looking for more work done with ToonLoop (there’s not much out there just yet), I came across another creation by Joy Penroz. It’s not a stop motion performance, but it runs with parallel ideas, looping to manipulate time in a modern pop take on the work of Dutch master painter Jan Vermeer. The contemporary “Milkmaid”:

THEMILKMAID from Joy Penroz on Vimeo.

Extra thanks to Michela Ledwidge and Austin Gambles on Twitter.

Multitouch Evolution: Free PyMT Framework, in Action

Puddle of Life - Darwin Exhibit from Tiago Serra on Vimeo.

Here’s a really elegantly-designed multi-touch table. It uses two really key pieces of open source technology: the Community Core Vision (“CCV”) tracker, formerly known as tbeta, and a lovely framework for coding visuals called PyMT.

PyMT, as the name implies, is a Python-based framework. I’ve gotten to know Nathanaël Lécaudé, a talented artist and coder who was nice enough to put me up a couple of nights while I was in Montreal; he’s one of several core coders. They’re doing a lot to really encapsulate functionality in widgets in a nice way. Features of PyMT include an event framework, specialized widgets for gesture, touch, and layout, and connections to OpenGL, OpenGL shaders, and sound. You can even work with the enduring, evergreen synthesis language Csound using its Python bridge, the oddly-named but powerful Ounk.

Of course, that’s all plumbing. It’s nice to see this applied to something powerful and educational. From the Vimeo description:

"Puddle of Life" is an educational multi-touch installation that was designed for Coimbra’s Science Museum (Darwin exhibit), demonstrating the theory of natural selection, as part of Darwin’s 200th anniversary.

The installation is composed of a round multi-touch table whose surface represents a virtual environment where 4 species of creatures live in. Each of these little creatures have different physical characteristics visually expressed by it’s quantity and/or size: Vision, Locomotion and Fur.

The player interacts with the game by touching the surface. He has to maintain full awareness of his creatures emotions (visually represented by a cartoon like ballon) and using this information to properly choose the mating partners on the control console. This console also allows the user to select the most appropriate descendent from 4 possible mutations, resulting from the reproduction.

The player’s objective is to help the species he controls achieve the highest number of creatures of its kind. Since this world suffers from climate change the player must assure that his creatures are well adapted to this ever-changing environment by choosing the best balance between mating partners and descendants.

The species move in a swarm, but they split when the predator is near. When a creature sees food it warns the nearby siblings and they all run towards it. Of course only the ones who have best locomotion reach it. The vision gives them the ability to see the predator sooner and the ability to see the food further ahead too. The fur is useful to them when the temperature is low but harmful when it’s hot, leading the creature to a shorter lifespan.

The player wins when they reach about 18 living creatures and loses when all of it’s creatures die from cold/hot, predation, famine or old age.

Photos: flickr.com/photos/tserra/sets/72157619458503007/with/3619335559/

Technical Specs:
LLP multi-touch table
CCV tracker - nuicode.com/projects/tbeta
pyMT framework - code.google.com/p/pymt/
Cython - cython.org/
Rabbyt sprite library for pyglet - matthewmarshall.org/projects/rabbyt/
Animation and artwork: Adobe’s AE, Ai and PS with Fasticon.com icons

The project is the work of Tiago Serra and his collaborators, who have also been PyMT contributors (see comments). Great work, gang. We’ll be watching for more.