Our friend Andrew Benson got the attention of MIA here on Create Digital Motion with his real-time glitch creations in Max/MSP/Jitter. Andrew shares some stories from the road with a detailed gig report from Coachella, which reveals a bit of what goes on backstage at these shows. I also really enjoy this clips, because lots of techniques that were once typically pre-rendered or assembled as static motion graphics clips are increasingly applicable in real-time. That makes for an extended palette for visualists – and very good times ahead.
Here’s Andrew – a rough and uncut diary, but with lots of juicy details as a result. The big revelation: we need to get out there and evangelize doing things live, with artists major and obscure alike.
Jack Lykins sends us a really amazing video he assembled using Ms. Pinky, the vinyl control system, and its included Maxi-Patch Max/MSP/Jitter patch to control animation interactively. We’ve seen vinyl triggering and controlling video, of course – as on the Serato VIDEO-SL, previously reviewed here. But there’s something about Jack’s style of “narrative” animation on the turntable that’s really compelling.
Musician Christopher Willits has an ongoing series for XLR8R Magazine in which he talks his own technical workflow. In the latest episode, he adds live visuals to his Ableton Live set using Max/MSP/Jitter. What’s nice about this is you see how some clever mapping can make visuals integrate neatly with music.
I’m somewhat insane, so my own setup often involves simultaneously running visuals separately with no communication with my music software. That allows me to set up less-direct relationships between visuals and sound.
But, while the techniques could be combined to a variety of setups, this also serves as a nice introduction to how you might use patching in Jitter alongside your music software.
Curious to know what you think of the presentation and content here, as I hope we’ll do more videos like this ourselves.
Jitter works brilliantly when it comes to processing signal - and that means for signal-like work with video and textures, it’s fantastic, as well as the usual Max-y tasks like processing input from physical sensors and input devices and the like. But try to do a whole lot of sophisticated 3D work, and Jitter may not be the best tool. For game-style 3D graphics and interaction, you want some standardized rendering and scene graph tools to take care of the hard work, plus physics and other capabilities that bring together your 3D scene.
That’s why [myu], the Max - Unity Interoperability Toolkit, looks so appealing. It not only allows for bi-directional data integration (via TCP) of Max and the Unity game engine, but can dynamically pass textures between the two. For those of you comfortable patching, say, chains of shader processors in Jitter, that means you can very quickly add some of the tasty 3D scene powers of Unity. Put together your textures in Jitter, and, say, dynamically process input from a Wii Fit balance board, then bring the input data and textures into Unity. (Unity is a friendly, elegant game engine built in C# and Novell’s open-source Mono implementation of Microsoft’s .net. Unity had previously been Mac-only but with a major new release now runs on Mac and Windows.)
The toolkit is the result of research at Virginia Tech Interactive Sound & Intermedia Studio director Dr. Ivica Ico Bukvic.
Needless to say, this could have powerful implications for all kinds of live and interactive installation applications. And yes, it is all released under the GPL.
Yesterday, we saw some splashy video distortion techniques applied to real-time video. You know what that means: it’s time to use these in live performance.
Here’s one start.Peter Nyboer, Max whiz and Livid developer, has run with the idea of squishing around video using optical flow analysis, and shows you how to add the effect to Livid’s Cell DNA VJ app. For Jitter users, this means you can rely on Cell for quick access to video taps and files, while adding unusual effects built in Jitter to get your custom processing on, not only with this example but any other patches you’ve created. One little detail of Cell DNA I missed – it requires Max 4 patches, not Max 5 patches. Peter has also posted a tutorial for working with that, after the jump.
And yes, if none of this is really making sense to you, you can go download the files and just try it out – no need to fully grasp all of the internals straight away.
Don’t want to use Jitter and/or Cell? The guts of Andrew Benson’s video datasplooshing technique is an OpenGL (GLSL) shader, so it doesn’t even rely on Jitter – Jitter can just be a convenient environment for playing around with such things. There’s word we may see a Quartz Composer wrapper around this shader, which would make it easy to use with software like VDMX.
Oh, by the way, I’m officially rescinding my editorial ban on the term “datamoshing.” Why? Because it means absolutely nothing, and therefore can be declared reasonably harmless. Also, unlike the term “glitch,” it comes without any baggage. We therefore have a nice, nonsense term for making video all mushy and unpredictable – a good thing.
22
