Apple’s iPhone — and the significantly more affordable, doesn’t-have-to-be-a-phone iPod Touch — are essentially pocket-sized, intelligent multi-touch controllers. Hooking them up to visual software as controllers simply requires some app on the phone to transmit data, and some way of dealing with that data on the computer side. We’ve already seen this a bit on Create Digital Motion, and we’ve been covering some of the specifics of parsing data with Pd (Pure Data), the open-source, tri-platform patching software, on Create Digital Music this week.

Here’s the basic setup:

On Your iPod/iPhone

You have two options of software to use on your iThing. (You’ll need to “jailbreak” your device, as these are not — and may never be, for all I know — approved Apple apps.)

1. mrmr by Eric Redlinger of Brooklyn (top right):  open-source, editable control screens (requires Mac-only software to edit). See our interview with Eric, including some examples with Quartz Composer.

2. akaRemote.app by Masayuki Akamatsu of Japan: not open-source, not editable, but comes with a set of useful control templates, and you can transmit data to the app. See our look at a recent release. Upcoming Mac-only visualist app 3L has its own special akaRemote-based bridge called i3L, which also runs on iPhone/iPod Touch; see our look at i3L with artificial eyes.

On Your Computer

While the iPhone and iPod Touch have Apple logos on them, all of these apps send OpenSoundControl data. That means any OSC-compatible software will work, which is gradually including more visual software, as well as modular apps like Quartz Composer, Max/MSP/Jitter, Pd/GEM, and vvvv. (I love saying that last one … vvvvvvvvvvvvvvvvvvv. Okay, moving on.)

Of all of these, Pd is the one solution that’s free, open source, and runs on any platform. That means it’s also a viable candidate for translating incoming OSC data to more broadly-compatible MIDI. (i3L has you covered, as it uses a MIDI bridge.)

We have not one but two sets of tutorials / example patches for working with Pd on Create Digital Music, using a patch like Cesare’s, pictured above:

Control Music and Visuals with iPhone/iPod, Free Via Pd

Tutorial: More iPhone/iPod Touch Control With Open-Source Pure Data

So, Is It Worth It?

I usually don’t ask that question, preferring instead to report on what other folks are doing. But it is always worth asking yourself — and it is an entirely personal question. I’m not totally convinced in the case of these devices that I’d want to buy one solely for VJing, but then, what makes this so cool is that it adds on additional functionality to a device. (Too bad Apple is being so uptight about third-party development, but at least there’s an SDK — and plenty of hackers ready to break Apple’s rules.)

My own preference remains squarely with tangible controllers and tactile feedback, especially as some of the advantages of multi-touch are diminished by the iPod/iPhone’s diminutive size. But I absolutely see the argument for using these. What do you think, dear readers?

Space exploration has had a deep impact on the way a lot of us see the world and think about our art. So I can’t wait for this Saturday’s Yuri’s Night Bay Area, which will bring a convergence of bleeding-edge music, art, technology, science, and visualism to a 12 hour-long party on NASA’s airfield outside San Francisco. We’ll be bringing some of the best of this event to you all around the world. Read or subscribe to RSS for our special minisite to make sure you don’t miss a thing, wherever you are on the planet; updates will continue live through the event and in the couple of weeks afterward:

yuricdm.com: Yuri + CDM + music + motion
yuricdm.com RSS feed

If you’re near the Bay Area, let us know if you’re coming to the event. We’d love help with photos, video, and coverage. And if you’re involved in a Yuri’s Night somewhere else in the world, let us know about that, too.

Here’s just a sample of the visualist angle at Yuri’s Bay Area:

It’s no secret: the once wildly-popular ArKaos VJ software has been looking a bit long in the tooth lately. And upgrades only get you so far: sometimes, as software matures in its life cycle, you have to redo the foundations. That’s what ArKaos has been working on, and the results, renamed GrandVJ, are due out soon (quarter 2). The new version emphasizes a re-worked, cleaner interface, multi-threaded graphics, and lots of effects and generators.

ArKaos GrandVJ Announcement

It’s up against some stiff competition. There’s the ground-up rewrite VDMX5, with some powerful semi-modular capabilities, the beefed-up Livid Union 2.5, Modul8, Resolume, and new generative app 3L. (See why 2008 will be a big year for VJ software.) That said, of these, only Union is cross-platform.

One differentiating feature in GrandVJ, as partly inherited from ArKaos VJ before it, is its “synthesis mode.” This maps sources onto a virtual music keyboard; combined with effects and generators that could retain ArKaos’ place as an easy, instrument-like visual tool.

The generative idea is nice, as well, but even with Flash support, I wonder how ArKaos will hold up to other generative competition. 3L has powerful tools for building 3D graphics in the software, Processing is gaining support among coders, Quartz Composer has native support in software like VDMX for custom patches, and Salvation has its own generative tricks, to say nothing of Jitter, Flash/Flex, and vvvv. On the other hand, ease of use can make a big difference in the market, and ArKaos’ interface is uncommonly clean. And then there are the ArKaos loyalists, who could find a compelling, friendly upgrade. (If you buy ArKaos now or bought it after March 1, the upgrade is free.)

We’ll have more details when this ships.

The nicely-growing Digital Tools blog has an excellent interview with visualist Paris Treantafeles, who works with lo-fi 8-bit-style visuals using tools he’s built for GBA and the Linux-powered Gamepark.

Interestingly, while a lot of people will dismiss the 8-bit movement as “nostalgic” — implying it’s just 20-somethings pining for their Mario-playing childhoods — Paris’ inspiration was originally vintage analog synthesizers. And synthesizing graphics is his main interest:

I concentrate on creating graphics from scratch. That’s pretty much all I do. Other people like using movie clips and manipulating them, but from my point of view it’s a good exercise to see what you can do when you have to create everything from scratch. It gives you an appreciation to form and color.

Hally // Blip Festival 2007: The Videos from 2 Player Productions on Vimeo.

The synthesis/sampling argument I think is very much related to the way electronic music is produced. I find that focusing on either one can be a good exercise — see our friend Troels sampling Coke bottles, for instance.

It’s nice stuff, but I do hope, particularly here in the US where the VJ/visualist scene has had trouble gaining broader recognition, that we start to see other styles on genres forming more coherent “scenes” in the way 8-bit has. Of course, what has happened for people like Paris is he’s found strong advocates in the musicians, which seems to be a key element (and has helped strengthen the visual work done outside chiptune music, as well).

I used to be resistant to the idea of coding. It wasn’t just fear that I couldn’t do it, though that was part of it; it was also the sense that I wouldn’t be able to get to the actual art and music making if I got too involved in programming. And, actually, that bit can be true. But a group of pioneers, working on projects like Processing, OpenFrameworks, and other intelligent development frameworks, has been working really hard to make code an elegant an expressive tool rather than a hindrance. Processing has reached widespread popularity because it does this really, really well — even if you’ve never programmed before.

I’ll be teaching a three-part class on Processing at Harvestworks in New York next month. If you’re in the area, there should still be openings if you’d like to sign up (and if you’re enrolled, feel free to holler hi here — if I hear from you in advance, I can help tailor the course to your needs).

For intermediate digital artists, even those who have never coded before, we will introduce techniques in Processing. Processing is an elegant, high-level, Java-based tool designed to make coding friendly to artists. We will learn how to create generative art in just a few lines of code, building interactive works in minutes. We’ll also look at some of the deeper possibilities for manipulating data, video, images, sound, and MIDI and other I/O. The emphasis will be on basic sketches that help introduce fundamental coding skills.

Wednesdays, March 5, 12 and 19, 6:30 – 9:30pm
$325/$385

Class page / signup @ Harvestworks

The class will specifically focus on how to make video, 3D visuals, MIDI, and sound work for performance. Making Processing a performance tool definitely involves some particular skills. But I’ll also use this as an opportunity to teach very basic coding techniques so that unfamiliar programming topics can immediately generate something on the screen or some sound, since that’s part of the appeal of the whole tool.

But what if you’re not in New York?

We’ll soon have CDM Labs up, which will include examples from the team at CDM, plus other stuff from around the Web, not only in Processing but related tools, as well. I’ll use this as a playground for the course, so what I share with them, I can share with you. And, honestly, we hope this will help discipline us here to keep coding and keep documenting. More on that soon.

I’m also hoping to refine this course into something that can be offered elsewhere; if you’re interested, get in touch.

More on Processing:

Random sketchbook of mine, the kind of stuff you can put together in minutes

Flickr Processing pool

Processing videos on Vimeo

Processing tag on Create Digital Motion

Official Processing exhibition page

The Mac may be in the spotlight these days, but Windows may boast the broadest access to freeware and open source tools for live visuals.

The latest edition: VisualJockey, as pointed out (alongside other free Windows tools) by beatfix on comments.

VisualJockey: Real-time Animation

You get a pretty powerful set of tools in this app, first introduced in 1999:

• Full Windows support, including Vista
• Alpha support throughout; image, AVI, QuickTime file format compatibility
• Global keystone capability
• MIDI, multi-monitor support
• Compatible with FreeFrame plug-ins (open plug-in spec for visuals)
• Sound beatmatching, internal LFOs with lots of waveshapes
• Generators for particles, patterns
• 2D color filtering, effects, blue screen
• 20+ transitions or custom bitmap transitions
• 3D support for 3DS import, primitives, 3D animation
• Export to AVI (which means it can double as an editor)

In fact, VisualJockey’s approach I think is unique — a set of tabs controlling different approaches, a hybrid blend of other interface paradigms. Want a timeline? A reactive sound system? A modular, generative 3D patch? It’s all in there. The UI is decidedly retro, and you get more flexibility from true modular patching environments, but at this price, if you feel like you want another tool in your belt, it’s hard to resist. And with export, this could be handy to have around alongside your existing tool of choice.

But VisualJockey is just the start — here are a few more from beatfix (and me):

Fixate from Aaron Sjogren on Vimeo.

Aaron Sjogren shares this VJ test set from an installation last week. It raises an important point — even as we’ve talked about the importance of getting “out of the box,” rectangles can be cool, too. (Notice: combining multiple 4:3 rectangles already means you escape the bland aspect ratio.) And it illustrates the importance of good documentation, ideally in an actual environment. (Previously reliant on pricey cameras, cheap HD can give you great results if you know how to use it!)

I love the idea that communities like Vimeo will help raise the bar for visualists sharing their work.

Full HD version available on this video’s Vimeo page, along with more from Aaron.

And since Aaron wasn’t sharing, anyone care to ID the visual app in use on his Mac? (I also think people are warming to dudes standing behind computers, much as eventually we got used to people manning synths and mixers.)

In case you haven’t noticed, a theme has been emerging on CDMotion: alternative interfaces geared for performance, software that expands the range of expression, and projection that gets visuals out of the 4:3-ratio box. There’s no one solution, and the expense involved in the tools covers quite the gamut. (I may have to personally start the “ghetto” guide to this stuff.)

But bringing together these elements is still essential to raising the impact of visuals. I still hear the “but the visuals aren’t as important” over and over again from people outside our field, and I think it’s a phrase that is — mercifully — doomed to extinction as visualists expand their craft.

So, one artist collective we’ve been covering is Turkey-based artificialeyes.tv. We’ve seen the Video Moving System, the automated mirror system for projection, the Vixid mixer with its matrix and multiple blend mode features (among other things), and the eyeball-scorching interface of 3L (”Thrill”) now in beta.

To see it all put together, of course, we’d really like to hit the club in Istanbul. Barring that, though, Michael from artificialeyes.tv has directed us at some images and video.

It’s really clear to me from these images how little you need lights. And in motion, with architectural effects amplifying the imagery, the projector itself can be more than just lighting effect (especially with these in motion, something you don’t get from the stills).

As digital musicians have realized for some time, working with computers is all about physical control. It’s the difference between feeling like you’re operating software and playing an instrument. So it’s no accident that Jay Smith is quick to call the Ohm, a new hardware controller for visuals and music, an “instrument.” I got to hang out at the Hoboken, New Jersey office of Livid and play with the Ohm a bit. Hands-on experience is everything: as you can see, you’ve got a nicely-crafted wooden crossfader piece, for starters. Here are some first impressions.

I was first introduced to Cornelius in Film School, where the buddy I often shared a dark editing room would play ‘Count Five or Six’ while I attempted to count and mark 16mm frames on the tiny strips running through our moviolas. This January, I jumped on the chance to see him at the awesome Walt Disney Concert Hall with Plaid.

Let’s start off with Plaid. I was very excited to see them, after catching their AV Set with Bob Jaroc at the Natural History Museum last year. The visuals, unfortunately, didn’t do much for me. There were a few sweet moments, but mostly I felt trapped in a semi-monotonous 3d visualizer. As we broke for intermission, I talked with a few other VJs and they agreed. I was skeptical about Cornelius, and figured I’d just have to settle for a rockin’ audio set and not expect much from the visuals.

I’m glad I was wrong.

Cornelius’ set started off with slow-mo footage of a giant bubble floating in a park. I couldn’t tell if it was CG or footage, and it didn’t matter. With the beautiful swells of the music and the rainbow-riffing undulating soap mass onscreen, I was lifted out of my funk and ready for an awesome show.

Each song had its own specific video - and each one looked and felt very different from the last, which does wonders to avoid visual burnout. Ironically, the first visuals set where I was amazed by the variety of visual atmospheres was that first Plaid/Bob Jaroc show.

His LED light rig was simple and spectacular. He had several lighting stands with rows of LEDs, and custom patterns that matched or contrasted the video for each song. There were also strobes mounted on top of each pole which were used sparingly and to great effect to punch up the impact of the hardest-rocking songs. You can see them in the following video:

Cornelius Live 01-17-08 from momo_the_monster on Vimeo.

Please note that this footage, taken in video mode with a still camera from the very last row in the hall, does not nearly do justice to the quality and variety of video. That said, it’ll give you a taste of the night, which is better than nothing.

One question that remains after the show - how did they keep it all synced up? I suppose Cornelius could be playing to a click-track, but it all felt very freeform, so I’m inclined to discount that idea. My only guess at this point is a DVJ or something similar that played back the pre-composed videos, with a VJ speeding up or slowing down the playback to stay in sync - like a DJ would do to keep two records from trainwrecking.

If you have any favorite moments from the show, or ideas on how they put together the video, leave your mark in the comments.

Read the full interview with Cornelius (Keigo Oyamada) on Create Digital Music. He tells CDM:

For the live performance, it’s an important part as there’s not only the sound in live shows but the visual aspect. By creating visuals I think it helps to understand more about the music as my songs are in Japanese.