lights on from thesystemis on Vimeo.

Digital visuals are often confined to a screen or a panel of wall. So there’s something magical about projects that get an entire building as a canvas. “Lights” is a live audiovisual performance for the Ars Electronica museum in Linz, Austria. The facade has some 1085 LED windows, controllable in real-time. The performance involved coordinating these windows with broadcast music.

The work was put together, stunningly, in just three days. OpenFrameworks, the artist-focused, C++-based code framework for “creative coding”, became a critical part of the process, assembling all of the real-time visuals. Zach Lieberman, co-developer of OF, also worked on the project and describes its ingredients and team:

this project was made as a collaboration between 4 different folks,
including daito manabe (musician & hacker), damian stewart (artist and
one of the creators of rjdj), joel gethin lewis (formerly with united
visual artists, where he worked on projects like massive attack’s LED
show) and myself (developer of openframeworks).
–> (daito) [daito.ws]
–> (joel) [http://www.joelgethinlewis.com/]
–> (damian) [http://frey.co.nz/]

we did alot of stuff with software that might be interesting for your readers — the tools involved (abelton, max, pd, openframeworks, dmx) and the challenges of a display like that, etc….

Breakdown of the tools:

• OpenSoundControl (OSC) for connecting audio and visual elements (and as Zach and I discussed privately in an email, it’s really the power of being able to relate different media, physical, aural, and visual, that defines the project more than any one tool)
• Max/MSP and Ableton Live for the audio score
• Pd (Pure Data), Max’s open-source cousin, for recording audio and OSC control signals

Zach notes “what I liked about it was how eye opening it was to feel that you can use each tool for what it’s good for.” 

Interface 27 from CyberPatrolUnit on Vimeo.

There has been a long tradition in live visuals and motion graphics, inherited from many other media, of maintaining a “secret sauce,” or the guarded formula of eleven herbs and spices. Ironically, for all you hear today “DIY” and “open source” in the same sentence, a lot of the motivation for doing something yourself has historically been doing something no one else can. Keep your secrets, and raise your value.

As our friend Bryant Place / CyberPatrolUnit sends over this latest set of live clips from a recent gig, and I browse through the comments, and reflect on the conversations I had last week at OFFF and during and following my own talk there, though, I’m struck.

The world has changed. First off, the Internet isn’t really about secrets. Your value is almost in direct proportion to how much you can share. Connections are forged through links of mutual exchange and good will. It’s not just about sharing your output or getting fans (the MySpace model), but sharing with a network of enthusiasts, and fellow artists. Those are the people from whom you often get real support (artistic, technical, and personal), gigs – and inspiration. (Even if you hate 8-bit music, that community is a really amazing model: their work to support each other and advocate for the whole subgenre has been I think the single biggest ingredient in their viral success.)

The visualist community increasingly itches not only to improve the quality of their own individual work, but everyone around them. A lot of us are in a battle for the future of this whole medium. Some parts of the world are devoid of live visuals, while others have mass-produced club visuals filling the nightlife.

Before I get carried away, the video itself is just the latest from the ongoing Interface 27 series. It employs a touch interface to control abstract visual pictures formed from streams of particles.

The reason I’m pulling back into the larger question is that these visuals are enabled by a library for Processing, a library we’ve seen here previously, developed by Memo Atken:

MSAFluid for processing (and Java)

If you’d rather use openFrameworks, there’s that version, too, as pictured below running blazingly fast:

ofxMSAFluid for openFrameworks

There’s even an ActionScript 3 port, in case you want to code Flash on the beach.

ofxMSAFluid for openFrameworks from Memo Akten on Vimeo.

Magic is itself a kind of augmented reality, a willing suspension of disbelief as we watch what we know is a blend between what we’re seeing and what we simply think we’re seeing. We know it’s not all physically happening, but the act of seeing it is enough. So it’s fitting that someone would try to blend magic and augmented reality. The challenge is flirting with a taboo of the digital age: getting people to accept that digital magic can still be magical, and not just empty illusion.

Marco Tempest is a rare character who combines magic and technology. He’s an unabashed showman in the old mold: you know it’s a show, and you know he’s selling what he’s doing. He also has some terrific ideas, and he knows his tech. Marco’s sends us his latest augmented reality, which he says is entirely real-time.

And here comes the reversal of the usual magic trick. In the pre-digital age, you’d keep the illusion alive by sharing as little as possible. Explain the tools, and the trick may be ruined. But in the post-modern, post-digital world of sharing and open source, it’s the reverse. You actually need to share something in order to have credibility. (My guess is that this could be the trend for far more than just magic – look at the musician and visualist worlds.)

Marco writes:

this is 100% real-time stuff - No post-processing. Programmed In C++ with OpenFrameworks, OpenCV, ARToolkitPlus, MacCam and other Open Source goodies…

Being the fan of software pr0n that I am, I thoroughly enjoy the screenshots, which show off the OpenFrameWorks setup [site | cdmo tag]. Incidentally, you can use OpenCV not only with C++, but Java/Processing, as well – check out our tutorial. And Bryan Chung has been working on a version of the ARToolkit library for Processing, too. That’s not to take away from OFW – it’s a really powerful environment, and there remain advantages on the C side.

Here’s what it looks like behind the magic:

 More screenshots

ofxMSAPhysics v2 from Memo Akten on Vimeo.

In a word: yummy. Or maybe that’s “bouncy.” Memo Akten’s brilliant ofxMSAPhysics brings open-source physics programming to the C++-based OpenFrameWorks environment, an artist-friendly combination with an elegant API. Coding physics doesn’t require an advanced degree. See today’s post on the beautiful My Secret Heart to watch this library in eye-popping action.

• particles
• springs
• attractions (+ve or -ve)
• collision
• replay saving and load from disk (temporarily disabled in current alpha release)
• custom particles (extend ofxMSAParticle and add to the system)
• custom constraints (extend ofxMSAConstraint and add to the system)
• custom force fields (extend ofxMSAParticleUpdater and add to the system)
• custom drawing (extend ofxMSAParticleDrawer and add to the system)

ofxMSAPhysics Project Page

By the way, don’t assume that the C++ folks get to have all the fun. Aside from the relatively simple but fast-and-useful traer.physics which inspired this more sophisticated outing, Java has stuff like:

jBullet
jME Physics 2 for jMonkeyEngine [wiki link]

There’s also a java.net project to bring the Open Dynamics Engine to Java, though I’m not sure that that’s terribly current. Java folks, feel free to chime in. But then, if you’re looking to migrate from Processing to OpenFrameWorks, this looks tasty indeed.

Perhaps it’s the church setting for the installation, or the strains of 17th-Century choral composition by Allegri. But Streetwise Opera’s My Secret Heart, binding together reactive visuals and post-Minimalist musical strains, has the feeling of a 21st Century passion play, a digital devotional piece.

My Secret Heart is a commission by Streetwise Opera, which develops work in the UK’s homeless centers, featuring performers from those shelters as stars, then transposing them to venues like Westminster Abbey. This work is a collaboration between electronic composer Mira Calix (Warp) with sound designer Dave Sheppard, and directed by video artists Flat-e with custom software programmed by visualist Memo Akten.

An installation, a film, and a performance, My Secret Heart’s 100 performers are wrapped in a virtual sonic and visual world, interwoven with digitally-generated, flowing tendrils, and driven by software that responds to sound and user interaction.

HD video on Vimeo

Director Flat-e and visualist Akten used a variety of techniques to create the organic fabric of the visuals:

• Filmed live action sequences and rendered After Effects sequences
• Visual components built in Quartz Composer, the free Mac developer tool
• Custom C++/OpenFrameWorks software for live performance
• Artificially-intelligent flocking behaviors, forming the particle strands that wend their way through the piece, as initially prototyped in Processing
• A new, original particle/physics engine built for OpenFrameWorks. Originally inspired by the open source traer.physics library for Processing, Memo’s library will support advanced features like recording and playback and pluggable force fields. (That makes me think of the Death Star tractor beam and the Millenium Falcon, but that’s probably just me.)
• A multi-dimensional spline interpolator for translating the particles into the flowing tendrils of the piece, also built for OpenFrameWorks.
• Cylindrical projection on a 16m-circumference “aquarium” rig by Gaianova