When it comes to controlling software, let’s put it bluntly: OSC good, MIDI bad. With OSC, it’s possible to control the array of things software might do, with easy use of high-resolution data, descriptive names in plain English (or your language of choice), a path hierarchy that makes it easier to structure messages in modular software, and smart networking features that makes assignment and communication a breeze. With MIDI, um… well, prepare for lots of mucking around.
Happily, visual software developers proprietary and open source alike have done what music developers generally haven’t - embrace OSC. Thanks to the fact that this community is unburdened by tradition and commercial development tends to involve small, responsive teams, change hasn’t been so tough.
So, visualists, it’s time to reap the fruit of that development work, and make the live performance rig work the way you’ve always dreamt it should work. Our friend Gian Pablo (check out his fantastic blog) clues us in to some recent developments with Resolume 3 “Avenue.”
Yes, you read that right: realtime stop motion. While stop motion is, by definition, associated with a painstaking process of creating animation frame by frame, a free and open source tool takes a different approach. ToonLoop provides the usual stop motion tools for creating loops, but takes a live performance approach to the recording and playback process, so you can turn your stop motion into a performance. The creator brought up the tool Saturday at the Open Video Conference in New York and got just the reaction you’d expect - a few confused (if delighted) chuckles, and someone asking, “That must be … slow.”
Now, if the framerate is low, you have no one to blame but yourself.
For fans of animation and live visualism, though, this is a dream. The first build was in Processing for Mac and Windows, but a new version for Linux (which should also work on Mac) is built on Python (with PyOpenGL, PyGame, Video4Linux and — oddly — Pure Data for MIDI).
In fact, I’m not sure whether I should tell you to download the thing or just run with the idea itself. (There’s no reason Java/Processing shouldn’t still work, by the way, if you use the excellent GSVideo library - and OpenFrameworks and others could be likely candidates, too.)
The idea is brilliant - and yet more evidence that being a visualist can be a much broader category than simply being a “VJ,” with the two-channel mix paradigm the more conventional term suggests.
And performances evidently look like what you might expect. Below, Joy Penroz uses Toonloop in Mérida, Yucatán, México, via the ToonLoop site.
Bonus video: as I was looking for more work done with ToonLoop (there’s not much out there just yet), I came across another creation by Joy Penroz. It’s not a stop motion performance, but it runs with parallel ideas, looping to manipulate time in a modern pop take on the work of Dutch master painter Jan Vermeer. The contemporary “Milkmaid”:
We have the tools. We have the techniques. Now, what happens when you put technology for tracking physical objects into the hands of artists around the world?
On June 6, members of the CDM community joined in our first “global hackday,” assembling tangible interfaces on tracking tables. Martin Kaltenbrunner of reacTIVision and the reacTable joined us from Vienna, Austria, while Adam Kumpf of the OpenFrameworks-powered Trackmate and MIT Tangible Media Group chatted from Cambridge, Massachusetts.
Much of the day was about saying “hello, world,” and helping each other through getting cameras working, troubleshooting, and the like. But there was an extraordinary variety of ideas even in one day. I can only imagine where things might go from here. I can also see the tools people were developing as being expressive for live visuals and visual performance - and via a really cheap interface, too.
Some of the accomplishments of our first day:
Endless ideas: Drawing interfaces using objects, a floor tom as a housing, a musical instrument with soda bottles as the interface, a game with blocks featuring the Tokyo skyline, and others.
Troubleshooting data: Both the Trackmate and reacTIVision projects got lots of feedback about how people were using the tools, where tracking was and wasn’t working, and where people got stuck up. We also compiled lots of information on cameras, drivers, builds, and operating systems. I’m working with Adam and Martin to dig through a lot of this information so we can compile a really practical guide to make it easier for people to create their own projects.
Special guests: Marco had his augmented magic show and we had the beginnings of an interactive glove. Check out the video highlights to see what the NYC event was like.
noisepages for networking, and other tools: Livestream video was a bit of a mixed bag; I’m still looking for easier ways of doing that (both on the video shooting side and the streaming side.) Text chat was an easy win, though IRC can still be cumbersome; I’m looking into integrating standardized XMPP chat instead, and providing access via any client or a webpage. But the other big success story was that noisepages worked nicely for documentation; see the fritzcrate and i3games sites for great examples!
Building Communities Around the World
A real highlight to me was getting to hear from Valeria (jalea.tv) and Jose (Estado Lateral Media Lab), visiting New York from Argentina. They talked about what the scene is like in Buenos Aires, and touched on issues of community, learning, open source, and the multilingual world of coding. And they do some really beautiful and hip visual work, both commercial and experimental. We also wound up with a significant amount of the online chat being in Spanish. That to me is a healthy sign - “global” really doesn’t necessarily translate to “English.”
Hackday Results, and the Future
For more documentation, head to our noisepages site:
And, yes, this is only the beginning. My suspicion is that a single weekend would be enough to get workable tracking projects going - especially as we iron out some technical wrinkles. But we’d certainly love to do more of this, whether it’s another “official” hackday or simple an open lab with chat and sharing.
noisepages is still in alpha state, but we’re working aggressively to move forward to beta, and content placed there is safe and future-proof. (Most importantly, I’ve fixed the jpeg library on the server so that avatars work!)
Trackmate is the inspiration for this project, partly because - building on the previous success of ReacTIVision - they’ve done a good job helping make it clear how people can get started, even if they’re new to this.
The mouse is not all that interesting as an invention. When people first saw mice, in fact, they typically weren’t terribly impressed, and often simply went back to their preferred non-keyboard input, the joystick. But destroy the novelty of the mouse, give it to half the population of the world and wait a couple of decades, and fantastic things start to happen.
See also: the knob, which is basically a simple hack for changing resistance in a circuit.
So, what could happen if we take novel interfaces now and try to accelerate what you do with them? That’s what’s starting to take place with tangible, multi-touch, and augmented interfaces, with the help of shared code tools (OpenFrameworks, Processing, ActionScript), shared libraries and trackers (ReacTIVision, the TUIO protocol, and LusidOSC/Trackmate), and communities like the fantastic NUI Group.
But enough about reading about this stuff and/or working alone. We’re going to try a new experiment in which we get lots of folks building this stuff – experienced users, relatively inexperienced users, your friends – and getting as quickly as possible into the business of actually trying apps, especially for the visual and musical performance stuff that we love.
Now, you may not have folks near you who are comfortable with code or have any idea what the heck we’re talking about. But readers of CDM and fellow hackers will join up on the Internet leading up to and around June 6. We’ve got a nice, fast Internet connection in New York, and we’re setting up some tools to help us share video streams, code we create, and to allow informal text chat.
Here’s how to get involved and join us.
Visual inspiration from the Trackmate project.
Head to http://hackday.noisepages.com/ for all the details. (If you’re interested in experimenting with in-development noisepages blogs and networking features as you make stuff, you may – ahem – find that registration is open.)
Share, the thesis project of Yannick Assogba in the MIT Media Lab Sociable Media Group, is an interesting idea in coding: it’s basically a peer-to-peer sketchbook for creative code. All of your sketches are synced to everyone else’s sketches, and Share tracks the connections between users.
You get more from Share than you would from simply, say, sharing a Subversion repository. Share not only syncs code and changes, but also tracks each time you copy and paste code from elsewhere, so that code snippets borrowed from others can be traced through the people using the system.
Up to 30 people are now invited for an online competition using the tool.
The Share Experiment is an online competition/design-a-thon/hack-a-thon and exhibition that invites 30 participants form to use Share to make new creative works over the course of ten days. The theme of this competition is "Inspired By Pong". Though the final result need not be games, artists/hackers are invited to reinterpret and remix the concept of pong while at the same time being open to reinterpretations and sampling of their own work as it being created. The Share Experiment will run from June 5th - June 14th and we are inviting applications. There will be some prizes awarded to winners (including iPod Touch[es] and Arduino kits) and we have some interesting ideas about mechanics for awarding prizes!
It’s a very cool idea, but this does raise some questions about implementation. It’s too bad that Share can’t run as some sort of plug-in; it loses some of the functionality of the bare-bones Processing editor, let alone the capabilities of an IDE like Eclipse or NetBeans. If it used a standard IDE, too, it’d be easier to be “language-agnostic” as the creator suggests. (OpenFrameworks or Flash or Processing, it wouldn’t really matter.)
But as a concept and an experiment, this looks really fascinating. It should be interesting to see how people use the code. And will users in a “competition” do a lot of copying and pasting, or focus mainly on their own work?
Part of the reason I bring up this is that we’re interested on CDM in doing some shared work. “Share” I think would be too limiting; it’s back to the old-fashioned Subversion approach.
So, for instance, we’re organizing a hackday around tangible interfaces in June, the first of what I hope will be many more. We’ll have people working on it in person in New York, but also folks collaborating around the world online. I’ll post more details, but just to kick off the discussion:
5
