We’ve seen terrific iPhone / iPod Touch apps for MIDI and OpenSoundControl, including Mrmr running VJ apps, i3L outputing to MIDI, and free, cross-platform Pd tools. With these, you can run visual, music, and other apps. But the latest addition is a very polished-looking app dedicated entirely to DMX, the protocol of choice for automated lighting and certain motorized projectors we love so much.

Luminar is a DMX lighting control app for iPhone and Touch, running control data for DMX rigs over wifi. There’s a touch-enabled mixer, precise, per-channel control, and color manipulation with a Color Changer channel layout. It’s definitely geared for lights in a way that general-purpose control software is not. I feel slightly icky talking about “lights” on this site. (Hey, aren’t those the things that blow out our projections?) But on the other hand, this kind of control and better software is just the kind of tool that can help give us better control over live rigs – and it’ll certainly work for DMX-controlled visuals, too, or (if you’re lucky enough to be doing this) synced projected visuals and lights, not to mention the kind of lights we very much enjoy (like LED arrays).

Despite my complaints about trying to run Processing in JavaScript in a browser, the recent port of Processing’s syntax to JavaScript has a lot of potential. Part of the underlying lesson here: platform-independence is cool. So where better to show that off than the iPhone/iPod Touch: a showpiece for Apple’s proprietary, platform-specific goodies.

German user mqwaq has ported the Processing.js port to iPhone. But it’s not the iPhone part that makes this so excellent, necessarily. Even cooler:

• SpiderMonkey, Mozilla’s C implementation of JavaScript, runs underneath. That allows for greater speed, and –
• It doesn’t require a browser to run. And –
• The graphics engine is OpenGL ES, the cross-platform 3D spec for mobile/embedded devices.
• And then he tweaked the whole thing to make it still faster.

The beauty of this to me, as well, is that the results can do all sorts of wonderful stuff — but you could easily bring the results to other mobile devices, too. (Hello, Android!) I’d still like to see Java on the iPhone, but it’s just one piece of the picture: the larger picture is the ability to make art with code on any platform using free and open-source code that works really well.

And credit where it’s due: the other reason this works well is that Apple’s done such a good job of making the OS and hardware interface work. This demonstrates that having a real OS has power not only when you’re using vendor-specific frameworks from Apple, but cross-platform frameworks, too. Now, if only Apple weren’t so restrictive about what they let you run.

Via Christopher Blizzard, who found this by way of JavaScript guru John Ressig who did the JS port in the first place.

Apple’s iPhone — and the significantly more affordable, doesn’t-have-to-be-a-phone iPod Touch — are essentially pocket-sized, intelligent multi-touch controllers. Hooking them up to visual software as controllers simply requires some app on the phone to transmit data, and some way of dealing with that data on the computer side. We’ve already seen this a bit on Create Digital Motion, and we’ve been covering some of the specifics of parsing data with Pd (Pure Data), the open-source, tri-platform patching software, on Create Digital Music this week.

Here’s the basic setup:

On Your iPod/iPhone

You have two options of software to use on your iThing. (You’ll need to “jailbreak” your device, as these are not — and may never be, for all I know — approved Apple apps.)

1. mrmr by Eric Redlinger of Brooklyn (top right):  open-source, editable control screens (requires Mac-only software to edit). See our interview with Eric, including some examples with Quartz Composer.

2. akaRemote.app by Masayuki Akamatsu of Japan: not open-source, not editable, but comes with a set of useful control templates, and you can transmit data to the app. See our look at a recent release. Upcoming Mac-only visualist app 3L has its own special akaRemote-based bridge called i3L, which also runs on iPhone/iPod Touch; see our look at i3L with artificial eyes.

On Your Computer

While the iPhone and iPod Touch have Apple logos on them, all of these apps send OpenSoundControl data. That means any OSC-compatible software will work, which is gradually including more visual software, as well as modular apps like Quartz Composer, Max/MSP/Jitter, Pd/GEM, and vvvv. (I love saying that last one … vvvvvvvvvvvvvvvvvvv. Okay, moving on.)

Of all of these, Pd is the one solution that’s free, open source, and runs on any platform. That means it’s also a viable candidate for translating incoming OSC data to more broadly-compatible MIDI. (i3L has you covered, as it uses a MIDI bridge.)

We have not one but two sets of tutorials / example patches for working with Pd on Create Digital Music, using a patch like Cesare’s, pictured above:

Control Music and Visuals with iPhone/iPod, Free Via Pd

Tutorial: More iPhone/iPod Touch Control With Open-Source Pure Data

So, Is It Worth It?

I usually don’t ask that question, preferring instead to report on what other folks are doing. But it is always worth asking yourself — and it is an entirely personal question. I’m not totally convinced in the case of these devices that I’d want to buy one solely for VJing, but then, what makes this so cool is that it adds on additional functionality to a device. (Too bad Apple is being so uptight about third-party development, but at least there’s an SDK — and plenty of hackers ready to break Apple’s rules.)

My own preference remains squarely with tangible controllers and tactile feedback, especially as some of the advantages of multi-touch are diminished by the iPod/iPhone’s diminutive size. But I absolutely see the argument for using these. What do you think, dear readers?

It’s not the lumens that count; it’s how you use them. But it’s easy to forget that when your projections just got blown out by lights, which someone used because they can move and your projection can’t. And it’s easy to get frustrated with the limitations of projection when you’re again looking at a static 4:3 rectangle on another flat wall.

Unfortunately, the art of using mirrors and other techniques to make projection more dynamic aren’t nearly as well known as they could be. Alternative projection techniques have also tended not to be productized. One significant exception is VMS or VideoMovingSystem. It’s the rare case of a hardware product made specifically for creative, live, performative projection. VMS is similar to the iCue moving mirror and some other tools, but it takes the kind of tools previously customized for lighting and specially adapts them to projection. You can actually buy a VMS unit with a projector already attached, or buy a unit that will fit a standard projector, making these more effective and easier to mount and use than lighting-specific instruments. It’s not a cheap solution for an independent VJ, but it is cheaper than competing custom lighting solutions. And if you read this site, you should already know that digital, computer-powered projection can do all kinds of things boring motorized lights can’t.

artificialeyes’ Michael Parenti and Todd Thille have taken a unique role in both championing the VMS tool and developing custom applications for it, as well as rocking Istanbul with the results. Michael said repeatedly that it saved the whole act of VJing for him. We got to talk to Michael and Todd about VMS and why it’s important — and, better yet, we got to play with these units, remote-controlled by artificialeyes’ 3L software and Michael’s iPhone. Even if you don’t plan on picking up VMS yourself (or I should say, convincing a club to buy them for you), you can tell from the interview how much of a difference changing a projection technique can make — not lumens, and not content, the two things we often get hung up on.

Jaymis: I have plenty more video from the ae guys waiting to be edited, both long-form looks into Thrill, and quick tips as well. That said, video is a bit of a new step for CDMo. This past year we’ve been talking about being a visualist mostly through the written word, so it would be great to get some feedback. Do you find video reviews and articles useful? Like the editing style? Think Peter should do voiceovers for software training videos? Hit the comments.

VDMX, the “realtime video studio” from the makers of Grid, semi-modular live visualist tool for Mac, and very possibly the “world’s most mature beta software”, continues to pack goodness into each new beta release. But the most recent addition, just now available as part of the primary public beta release, warms my heart. The latest beta adds support for OpenSoundControl or OSC, a protocol for communicating between computers, hardware, and software. With high-resolution data, an open-ended naming scheme, and network savvy, OSC is just plain better than MIDI at at least some tasks. But it’s suffered from a lack of documentation, a vague specification that’s a bit too open-ended and daunting to developers, and most of all, a lack of critical mass as far as tools. One small step at a time, that’s changing.

So far, the VDMX developers have tested their implementation with the multi-touch Lemur controller and inter-app data with Apple’s free Quartz Composer. But OSC is also supported in Processing, Max/MSP/Jitter, Pure Data, and free controller implementations for the Wii remote and Wacom tablet, among many other things. It’s also supported natively by the Monome controller, which is quite a lot cheaper than a Lemur. Want to pipe data from a Wacom into Quartz Composer? Sync data between a Mac running VDMX and a PC running vvvv? Something … uh … else that I haven’t thought of? Quite a lot is possible.

And an iPhone implementation could come next. (Or you can send OSC from iPhone right now, with MrMr.)

You can try this out in the beta that’s available now, though David Lublin from Vidvox warns us:

…there’s still more to do with OSC, currently no way to trigger files, it’s just control data, but the core functionality for sending and receiving is there, so we really just need to start building on top of that for whatever protocols there are to support

…the OSC support is still very much in it’s early stages.. it is there and stable, but we are barely scratching the surface of what we can do with it

Of course, that description could apply to OSC in general, which is why the two CDM’s (createdigitalmusic and createdigitalmotion) will be working over the course of 2008 to work with users and developers to share knowledge and get better implementations in everyone’s hands sooner. It’s something I hope to check in on while I’m at NAMM next week. Stay tuned.

Vidvox + VDMX5

VDMX5 public beta direct download link (Mac only)

Introduction to OpenSoundControl [opensoundcontrol.org]

Speaking of multi-touch OSC implementations, we hope to catch up with CyberPatrolUnit, who’s working on a VJ interface on Lemur, as pictured below. He’ll be bringing it to the CDM NAMM party, so if you’re in LA, come see us next Friday!

iPhone development continues despite a lack of SDK. Masayuki Akamatsu has updated his native iPhone OSC client aka.iphone to 2.1, and includes plethora of screen/input configurations to suit your mobile performance needs. The update should suit typical VJ and Dj applications with its range of GUI sliders, knobs, button grids, etc.

If you have an iPhone or iPod Touch, be sure and check out aka.iphone. Very cool stuff. Yes, sorry, another Apple oriented update. Shoot me.

[Ed.: Note you will want Max/MSP/Jitter to work with this app, though I think it should work with Windows, too; I just lack a machine that can test it. Be sure to see also Masayuki's complete Max collection, which now covers everything from Wii remotes to 3D SpaceNavigator controllers, Apple Remotes, and even the motion sensor in the MacBook Pro. Completely brilliant. And needless to say, even if you don't want to hop on AT&T's mobile network, this gives you options with iPod Touch, too, even if you can't play Phase on that. -PK]

Mobile, touch control of visual apps? We’ve seen one way to do it; here’s an even more compatible MIDI approach. I think I’d stick to the iPod Touch, personally, just to conserve battery on a phone. But it’s interesting. -PK

Hot on the heels of Mrmr, ArtificialEyes (the Istanbul-based VJ collective) have released i3L MIDI Bridge for the iPhone, which builds on Masayuki Akamatsu’s aka.remote.app, giving those without Max/MSP skills a simple way to bridge their iPhone and audio/video performance software of choice.

i3L (pronounced “i thrill”) is Freeware, and was developed using Max/MSP from Cycling74 and is a support application for aka.remote by Masayuki Akamatsu i3L receives pre-defined UDP messages from aka.remote.app running on the iPhone, scales the values to MIDI, and allows you to configure the sending MIDI channel and control change message number. While this software was developed to work seamlessly with our Real-time 3D VJ software Thrill, you can use this program with any audio or video software which receives midi messages.

If you give i3L a try, let us know how it works out for you in the comments.

artificialeyes.tv
aka.objects by Masayuki Akamatsu — must-have stuff for Max/MSP/Jitter!

Ed.: Does anyone know if multi-touch gestures would be possible in DIY apps? Maybe in February when Apple shares the official SDK? -PK

Click to play

Mrmr is an open protocol for mobile devices. It is used to dynamically create user interfaces on your iPod Touch or iPhone which respond to client apps in a multi-user performance environment.

Okay, that sounds awfully dry. Let’s try that again.

Mrmr lets you control Quartz Composer applications (or really , any compatible OSC implementation)over Wi-Fi from your iPod Touch or iPhone. Now you, too, can dance around like a lunatic while still controlling your visuals from the dance floor. Did we mention it’s multi-user, as well?

Mrmr is the brainchild of Eric Redlinger, researcher-in-residence at Brooklyn Polytechnic University’s Integrated Digital Media Program. He has leveraged the iPhone’s OS X underbelly and 10.5’s new Quartz Composer features to allow this sort of functionality.

I had the lucky* chance to interview Eric and ask him a few questions about Mrmr and the iPhone. Apologies for the quality of the interview, it was very spur of the moment.

Mrmr is a work in progress, but I think the results so far speak for themselves.

*(ok, no so lucky, my desk is right next to his, but somehow I had not seen Mrmr in action until just recently…)

No doubt you are aware of Apples updated iPod line, with the new flagship iPod Touch bringing the iPhone’s touch screen interface to more affordable waters. However, what I find interesting is that every iPod now supports 640×480 video playback and output, and component video output via the new Universal Dock and AV Cables*.

While compressed video with component cables seem like conflicting ideas**, the iPods support fairly high bitrate h.264 and MP4 video. With the new iPod Classics sporting up to 160GB of storage, thats a serious VJ Clip library in your pocket. Add on something like Karl Klomps Dirty Video Mixer and you have an incredibly porable rig. And with two iPod Nanos you might very well have the worlds smallest VJ Mixing rig?

Karl Klomps Dirty & Cheap Non-Sync Video Mixer. Awesome.

While not the most versatile setup, it seems a rather tempting solution just for novelties sake. What seems more fun, two iPod Nanos and a tiny homebrew glitch video mixer in your pocket or a V4, two laptops and a camera, and maybe a triggering midi device. Hm.. The Component video also gives you access to the RGB (or possibly YUV) color information, allowing for some interesting mixing should you feel the urge to get creative. Speculation aside, Portable Media Players are looking quite intriguing for VJs lately. That iPod Touch is looking awfully tempting, especially with homebrew applications and OpenGL acceleration.

Are any Create Digital Motion readers currently using some sort of Personal Media Player or Video iPod for clip triggering in sets, or other similar creative solutions?

*sans shuffle, but, its never really counted now has it?
** Component video cables output a cleaner signal, while highly compressed video is not pristine. If you have highly compressed video, you probably aren’t worried about quality.

Yes. You can now VJ from your iPhone, controlling your Max/MSP/Jitter patch via Safari and the new aka.iphone object from Masayuki Akamatsu of aka.wiiremote fame.

The aka.iphone object is a work in progress, but stay tuned, there’s much fun to be had. Can anyone say “pocket Lemur”?