Apple’s iPhone — and the significantly more affordable, doesn’t-have-to-be-a-phone iPod Touch — are essentially pocket-sized, intelligent multi-touch controllers. Hooking them up to visual software as controllers simply requires some app on the phone to transmit data, and some way of dealing with that data on the computer side. We’ve already seen this a bit on Create Digital Motion, and we’ve been covering some of the specifics of parsing data with Pd (Pure Data), the open-source, tri-platform patching software, on Create Digital Music this week.

Here’s the basic setup:

On Your iPod/iPhone

You have two options of software to use on your iThing. (You’ll need to “jailbreak” your device, as these are not — and may never be, for all I know — approved Apple apps.)

1. mrmr by Eric Redlinger of Brooklyn (top right):  open-source, editable control screens (requires Mac-only software to edit). See our interview with Eric, including some examples with Quartz Composer.

2. akaRemote.app by Masayuki Akamatsu of Japan: not open-source, not editable, but comes with a set of useful control templates, and you can transmit data to the app. See our look at a recent release. Upcoming Mac-only visualist app 3L has its own special akaRemote-based bridge called i3L, which also runs on iPhone/iPod Touch; see our look at i3L with artificial eyes.

On Your Computer

While the iPhone and iPod Touch have Apple logos on them, all of these apps send OpenSoundControl data. That means any OSC-compatible software will work, which is gradually including more visual software, as well as modular apps like Quartz Composer, Max/MSP/Jitter, Pd/GEM, and vvvv. (I love saying that last one … vvvvvvvvvvvvvvvvvvv. Okay, moving on.)

Of all of these, Pd is the one solution that’s free, open source, and runs on any platform. That means it’s also a viable candidate for translating incoming OSC data to more broadly-compatible MIDI. (i3L has you covered, as it uses a MIDI bridge.)

We have not one but two sets of tutorials / example patches for working with Pd on Create Digital Music, using a patch like Cesare’s, pictured above:

Control Music and Visuals with iPhone/iPod, Free Via Pd

Tutorial: More iPhone/iPod Touch Control With Open-Source Pure Data

So, Is It Worth It?

I usually don’t ask that question, preferring instead to report on what other folks are doing. But it is always worth asking yourself — and it is an entirely personal question. I’m not totally convinced in the case of these devices that I’d want to buy one solely for VJing, but then, what makes this so cool is that it adds on additional functionality to a device. (Too bad Apple is being so uptight about third-party development, but at least there’s an SDK — and plenty of hackers ready to break Apple’s rules.)

My own preference remains squarely with tangible controllers and tactile feedback, especially as some of the advantages of multi-touch are diminished by the iPod/iPhone’s diminutive size. But I absolutely see the argument for using these. What do you think, dear readers?

Daniel Wilcox is “Robot Cowboy”:

“A futuristic expatriate American Space Ranger combines wearable computing, midi guitar, and live energy to wander the digi-range as a robotcowboy playing for dying astronauts. This is a lofi-guitar-compu-show with algorithmic balls from a DEVOspud, laptop-stomping idiot wearing exposed electronics.”

Check out Robot Cowboys YouTube channel, specifically : Video Killed the Radio Star Karaoke. In the “Future”, all VJs must wear helmets as awesome as Robot Cowboys. AND IT SHALL BE.

It does not get any better than this folks. I would attempt to write more, but words fail me. I can only utter “awesome” over and over.

See more at Robot Cowboy, check out his “Wearable Audio Visual Unit” project and Robot Cowboy @ myspace. Well played sir, well played.

After twenty years or so of music software, you’ll often find that what you need to do is creatively satisfied by what’s available in pre-built tools. But video is often another matter: even if a VJ tool does what you need some of the time, there are times when you need something that doesn’t exist (or even something simpler, to perform a specific task). You might not be ready to invest hundreds of dollars in Max/MSP/Jitter to do that, of course. Enter Pure Data, Max’s free, open-source cousin. Like Max, it lets you quickly build custom tools by visually “patching” objects on-screen — ideal for modular video tools. But unlike Max, it costs nothing, it runs on Mac, Windows, and Linux, and because of the rich Pd community and a good relationship with the Max community, it has some tricks of its own.

Beau, aka DJCypod, writes to share his excellent video tutorial on building a simple 2-way mixer in Pd. Beau, if you ever need a second career, I think you could do relaxation recordings; I felt my blood pressure dropping. Next time I have students getting nervous about Pd or patching, I’m sending them to this video. (Well, and also because it’s really easy to follow.)

Totally beginner-friendly, so newbies will get rolling fast. And as a long-time Max user, it served as a nice introduction for me, too.

Pure Data Video Mixer: YouTube, Internet Archive

You’ll also need the Pd Extended Release; see our previous story

I’ll be building some Pd software mixers this month, so I’ll be sure to report back! (Any requests, or tips from those of you who know Pd better than I do?) Now, if only I could easily integrate Processing with Pd, as you can with Max. (At least Pd does have Java support of its own.)

Open Source software can be hard to approach sometimes, especially DIY programming environments, so there is some good news for those wanting to get started with two leading open source packages for performance.

Pure Data - the open source dataflow/patching environment now has a shared playlist on youtube. If you’ve been interested in Pure Data but hadn’t the slightest clue what it is or how to use it, you should check out the Pure Data Videopedia. The videopedia contains performance clips, tutorials and examples made with PD and some video related plugins.

Not into datalflow/graphical programming environments? Get your hands dirty with Processing. Check out the recently posted tutorials by user b2kn : “Coding for visual performance” workshop. You can find the accompanying processing sketches here. Good stuff!

Pd, aka Pure Data, is the free and open-source cousin of Max/MSP/Jitter. It’s powerful — even sometimes having technical advantages over Max — but has suffered from complex installation and dependencies, poor documentation, and an unpolished interface. Enter Pd-extended, a distribution that fills in those gaps. Pd-extended’s maintainer Hans-Christoph writes up what this is all about in an introduction on Create Digital Music, friendly even if you’re new to the Pd world.

Pd, Max’s Free Cousin, Gets Polish and Ease in Extended Build

Visualists should be especially interested in this latest release, because it offers much-improved out-of-the-box support for custom-patched 3D and video — especially if you’re on a Mac, for the PiDiP (though there are Windows and Linux improvements, too, and GEM works even with Windows).

Mac OS improvements:

• Image and video-processing PDP/PiDiP work out-of-box
• Anti-aliasing of boxes and lines in the interface
• New, purty icon

Linux:

• A .deb package for Debian and Ubuntu, with GNOME menu support. (`Bout time! Wonder if this means we’ll see it in the big Ubuntu repositories?)
• New icon

All platforms:

• GEM, the quasi-Jitter-like 3D and pixel library, has working shader support. Ed.: Truly outrageous.
• New libraries: mapping, msd, mrpeach net/OSC, flib
• [comport] is robust on all platforms (can you say Arduino?)
• Font-face and -font-weight command line options
• New font and layout is the exact same size on all platforms to the pixel. (previously you’d see some serious cross-platform glitching)

Pd Extended Release

And lest you have a bad taste in your mouth from the fugly older releases of Pd, Anton (vade) sends along the picture at the top of this story, showing the new UI from the Pd 0.40 dailies running on Mac OS X. Anton is also working on porting some of his brilliant visual patching from Max to Pd — and he’s living proof that even a Max die-hard can find at least some use with Pd, too. (The two environments are really, really close — sometimes confusion switching between them is because they’re so close, the differences can be confusing.)