Visual patchers love the fact that they can create sophisticated stuff just using on-screen patch cords. But when you want to go beyond the capabilities of the default set of objects — and for solving certain problems more elegantly — code is the way to go. Code your own objects, and you get the best of both worlds. So the news that vvvv is adding a facility for easily creating your own objects (or "nodes" as they’re called in vvvv-speak) is good news, indeed. (Jitter and Pd have similar extensibility, though having looked at what’s necessary to write a Jitter object, you may or may not want to … go there. This does look quite a bit easier, for simple tasks, at least.)

Phl shares some additional resources in comments on our previous story, so I want to make sure you don’t miss these.

sourceforge site with the first plugins (go to code -> svn browse)

VVVV Plugin HowTo

Plugin Forum for more questions

Note that you can use any language you like. I do like the open source SharpDevelop they mention here. For any of you who think Windows doesn’t have a fledgling open source community of its own, think again — perhaps because of the glut of working Windows-based programmers, there’s actually quite a lot going, arguably more than on Mac. And C# is a nice language for this kind of task.

Seriously, let us know if you build anything; we’d love to see it!

The free (for non-commercial use), Windows-only, insanely-powerful visual/multimedia patching environment vvvv just got a killer feature: the ability to make your own "nodes" (what are called objects in some other modular tools).

It’s actually pretty easy to build, too, and the developers have some templates for you. This is way up the list on my summer projects. You can do matrix transforms, as well, so 3D / GPU-based video processing gets very interesting, along with simpler manipulations of color, string, and number values:

The latest vvvv version offers developers an interface to write their own nodes for vvvv. A plugin is basically a .dll file, that can be drag&dropped into a vvvv patch where it appears as a node. If its stored in vvvv’s plugins directory, its even available in the node list.

vvvv gurus, if you give this a try and make something interesting, let us know:

vvvv40beta16 release with plugin interface [Results in Reverse blog]

Apple’s iPhone — and the significantly more affordable, doesn’t-have-to-be-a-phone iPod Touch — are essentially pocket-sized, intelligent multi-touch controllers. Hooking them up to visual software as controllers simply requires some app on the phone to transmit data, and some way of dealing with that data on the computer side. We’ve already seen this a bit on Create Digital Motion, and we’ve been covering some of the specifics of parsing data with Pd (Pure Data), the open-source, tri-platform patching software, on Create Digital Music this week.

Here’s the basic setup:

On Your iPod/iPhone

You have two options of software to use on your iThing. (You’ll need to “jailbreak” your device, as these are not — and may never be, for all I know — approved Apple apps.)

1. mrmr by Eric Redlinger of Brooklyn (top right):  open-source, editable control screens (requires Mac-only software to edit). See our interview with Eric, including some examples with Quartz Composer.

2. akaRemote.app by Masayuki Akamatsu of Japan: not open-source, not editable, but comes with a set of useful control templates, and you can transmit data to the app. See our look at a recent release. Upcoming Mac-only visualist app 3L has its own special akaRemote-based bridge called i3L, which also runs on iPhone/iPod Touch; see our look at i3L with artificial eyes.

On Your Computer

While the iPhone and iPod Touch have Apple logos on them, all of these apps send OpenSoundControl data. That means any OSC-compatible software will work, which is gradually including more visual software, as well as modular apps like Quartz Composer, Max/MSP/Jitter, Pd/GEM, and vvvv. (I love saying that last one … vvvvvvvvvvvvvvvvvvv. Okay, moving on.)

Of all of these, Pd is the one solution that’s free, open source, and runs on any platform. That means it’s also a viable candidate for translating incoming OSC data to more broadly-compatible MIDI. (i3L has you covered, as it uses a MIDI bridge.)

We have not one but two sets of tutorials / example patches for working with Pd on Create Digital Music, using a patch like Cesare’s, pictured above:

Control Music and Visuals with iPhone/iPod, Free Via Pd

Tutorial: More iPhone/iPod Touch Control With Open-Source Pure Data

So, Is It Worth It?

I usually don’t ask that question, preferring instead to report on what other folks are doing. But it is always worth asking yourself — and it is an entirely personal question. I’m not totally convinced in the case of these devices that I’d want to buy one solely for VJing, but then, what makes this so cool is that it adds on additional functionality to a device. (Too bad Apple is being so uptight about third-party development, but at least there’s an SDK — and plenty of hackers ready to break Apple’s rules.)

My own preference remains squarely with tangible controllers and tactile feedback, especially as some of the advantages of multi-touch are diminished by the iPod/iPhone’s diminutive size. But I absolutely see the argument for using these. What do you think, dear readers?

video mlr clone demo from themoves on Vimeo.

I’ve always been intrigued by the idea of using the Monome, the popular open-source, button-bestrewn controller device, for visuals and not just sound. But I haven’t seen any examples in person. Via comments on Monday’s Live + Isadora story, here’s one example by Joshua Nugent. The software is inspired by MLR, the sound sample-rocking patch that comes with Monome. In fact, that’s one area in which I think the Monome hasn’t gotten enough credit — a lot of its popularity actually comes from its software design, not just its elegantly-minimal hardware design.

I tend to do visuals differently, so I’ll have to finish my kit, finally, and post some software examples. And yes, while this is done in Isadora, there’s nothing saying you couldn’t use other software tools.

Bonus points for including Bush dancing in Africa. Now someone send poor Josh a screen and some windowblinds.

More on this project on the Monome forums, including instructions:

how about a video version of MLR/64step?

I’d love to see more Monome examples, so if you’ve got them, send them in!

Gavin Morris has been working on an audiovisual setup with Ableton Live and Isadora, a tasty combination for any Windows or Mac user. Isadora, for those of you who don’t know, is a visually-focused modular patching tool. It’s similar to tools like Max/MSP/Jitter, but by emphasizing the practical needs of visual performance, it’s unusually usable when putting together real-world gigs. Its use by A/V dance troupe Troika Ranch (co-founder Mark Coniglio is also the tool’s creator) has also popularized it in modern dance circles.

Gavin has two tutorials for us to start. The first syncs up Live and Isadora, along the lines we ran here using Live by momo the monster:

AV Cutup Secrets: Using Lucifer & Live

Gavin writes:

It’s similar to Momo’s recent Tutorial but uses a free tool for the VST (Pluggo) and allows control from the Live interface (as opposed to within the VST) This allows you a lot more flexibility and means you can use Follow Actions, adjust loop lengths/positions in realtime and even create a slicer. It is Live>Isadora via OSC but could equally be to many other softwares and could equally use MIDI.

I’ve written a VST to go in slicer channels tool.

Sync Ableton Live to Isadora using a Pluggo VST from digital funfair on Vimeo.

Gavin warns us that the video may "put us off." At first I thought that meant it was NSFW or something, but … well, that’s not the problem. You’ll see. I leave it up to you to decide how you feel about it.

The second tutorial gives you the power of Emergency Broadcast Network-style A/V slicing:

I’ve done a tutorial for a Video Slicer - synching up Live’s slicer to Isadora - same technique but a bit of maths to convert the midi notes Live creates to video position. You can make some quite glitchy s***!

AV Slicer Tutorial - Ableton 7 Slicer with Isadora from digital funfair on Vimeo.

Lots more information at Gavin’s site, Boredbrands Digital Funfair.

He needs someone to build the Mac plug-in, so Max users, if you’re game, go for it!

AV Sync Tutorial

AV Slicer Tutorial

Good as this is, I hope we see some audiovisual setups that work with more asynchronous relationships between music and motion — I know my own tastes for my personal work tend in the abstract. Maybe I’ll have to put my money where my mouth is and write it up myself.

Cat hit up the Amoeba Dance comments with a link to Sanch TV’s work in vvvv.

Apart from some smooth motion and subtly textured shapes, Sanch is also collaborating on an AV act “Va”, with quad-screen visuals:

I’m sure I’m not the only visualist to have been inspired by Autechre’s Gantz Graf video, nor the only person to have watched it and though “some day, we will be able to do that in realtime”:

I think we’re still quite a way off, but the latest project to set my mind thinking along these lines is Amoeba Dance - Caliper Remote (and the followup, Amoeba Dance with Mad Girls,) by CDMo reader Memo:

This is created realtime in VDMX from a quartz composer generator, controlled by 9-band audio analysis, and topped off with a very nice little effects chain.

A few people have mentioned they don’t get the same look when they use the QTZ file, this is because the QTZ file renders with very basic shading and there is quite a bit of post done in VDMX. The Effects I’m using (from top to bottom) are:
Serpia Tone (100%, Source Atop)
Shaded Material (0-40% tied to audio analysis, Soft Light)
City Lights (100%, Source Atop)
Bloom (100%, Screen)

I’ve also got a 9-band audio analysis going on, with different frequencies driving all the parameters of QTZ. You can just setup the frequencies randomly and it will do pretty cool stuff to almost any song (see http://www.memo.tv/amoeba_dance_v1_5 for an example!!), but it is best to taylor the frequencies to the specific song…

Memo shares the GLSL code and the QTZ file on his site, which contains some interesting nuggets of QC, Actionscript, Processing and other codey goodness. PK: Because this uses GLSL which runs in any OpenGL environment, you could also port the geometry stuff to Processing, Max/MSP/Jitter, Pd/GEM, or (with some adjustments) even things like vvvv, etc. — no need for Quartz Composer per se.

He also maintains a VDMX and Quartz Composer repository: http://vdmx.memo.tv/.

Awesome work, and more to come it seems.

For all of the other CDMo readers who are doing cool things, don’t wait for us to find you: Hit the comments or the contact page and tell us what you’re up to!

It’s still early in development (read: it often crashes), but The Bean Machine applies nodal, patch-based development to Java. The interface is mysteriously close to Quartz Composer, down to capabilities, UI, and even the 3D cube tutorial. Personally, I use Java because it can do things Quartz Composer can’t, but it’s interesting nonetheless — and raises, again, the question of why we don’t see more tools that try to meld the capabilities of code and patches.

The cool bit: nodes are Java Beans, so you really could use this to combine the best of both worlds if it matures. No download yet, but we’ll be watching … perhaps it will inspire other developers, as well.

The project is labeled “experimental”, but could be worth a look. Developer Jerry Huxtable has lots of other goodies for Java-heads on his page, including lots of 2D image processing stuff and a map editor — Processing lovers, might want to pop this into your del.icio.us.

Bean Machine @ JH Labs

JH Labs main page with lots o’ projects

The Mac may be in the spotlight these days, but Windows may boast the broadest access to freeware and open source tools for live visuals.

The latest edition: VisualJockey, as pointed out (alongside other free Windows tools) by beatfix on comments.

VisualJockey: Real-time Animation

You get a pretty powerful set of tools in this app, first introduced in 1999:

• Full Windows support, including Vista
• Alpha support throughout; image, AVI, QuickTime file format compatibility
• Global keystone capability
• MIDI, multi-monitor support
• Compatible with FreeFrame plug-ins (open plug-in spec for visuals)
• Sound beatmatching, internal LFOs with lots of waveshapes
• Generators for particles, patterns
• 2D color filtering, effects, blue screen
• 20+ transitions or custom bitmap transitions
• 3D support for 3DS import, primitives, 3D animation
• Export to AVI (which means it can double as an editor)

In fact, VisualJockey’s approach I think is unique — a set of tabs controlling different approaches, a hybrid blend of other interface paradigms. Want a timeline? A reactive sound system? A modular, generative 3D patch? It’s all in there. The UI is decidedly retro, and you get more flexibility from true modular patching environments, but at this price, if you feel like you want another tool in your belt, it’s hard to resist. And with export, this could be handy to have around alongside your existing tool of choice.

But VisualJockey is just the start — here are a few more from beatfix (and me):

Vlad Spears, musician, programmer and maker of the Davel.Plugs audio plugins (previously reviewed on CDM) has a write-up about the forthcoming Max/MSP 5. Andrew Benson of Cycling ‘74 gave a presentation at the Bay Area Computer Music Technology Meetup and Vlad kindly drew up a report. Most of the features he describes will not be new to those who’ve been awaiting Max 5, with the possible exception of:

- Say goodbye to [prepend set] messages! Message boxes have a right inlet specifically for this function.

- Multiple live views on the same patch, at different magnification levels. As someone who often builds patches which spill beyond the bounds of a screen, this is fantastic. Changes update in all views simultaneously.

- Object name auto-completion. To quote Andrew, “Several of our developers bought iPhones during development and fell in love with auto-completion.” Arrow through the drop-down list of auto-complete object choices and the Clue window shows information and arguments for each.

The grapevine says that the public beta of Max 5 will drop within several weeks. /me twiddles thumbs…

Ed.: Word we got from Cycling ‘74 at NAMM was that you’ll have the final version by the end of first quarter. I’m actually wondering if they’ll either forgo public beta — or slim it down in length — but that means you’re still just a few weeks away from the next release of Max.