Despite my complaints about trying to run Processing in JavaScript in a browser, the recent port of Processing’s syntax to JavaScript has a lot of potential. Part of the underlying lesson here: platform-independence is cool. So where better to show that off than the iPhone/iPod Touch: a showpiece for Apple’s proprietary, platform-specific goodies.

German user mqwaq has ported the Processing.js port to iPhone. But it’s not the iPhone part that makes this so excellent, necessarily. Even cooler:

• SpiderMonkey, Mozilla’s C implementation of JavaScript, runs underneath. That allows for greater speed, and –
• It doesn’t require a browser to run. And –
• The graphics engine is OpenGL ES, the cross-platform 3D spec for mobile/embedded devices.
• And then he tweaked the whole thing to make it still faster.

The beauty of this to me, as well, is that the results can do all sorts of wonderful stuff — but you could easily bring the results to other mobile devices, too. (Hello, Android!) I’d still like to see Java on the iPhone, but it’s just one piece of the picture: the larger picture is the ability to make art with code on any platform using free and open-source code that works really well.

And credit where it’s due: the other reason this works well is that Apple’s done such a good job of making the OS and hardware interface work. This demonstrates that having a real OS has power not only when you’re using vendor-specific frameworks from Apple, but cross-platform frameworks, too. Now, if only Apple weren’t so restrictive about what they let you run.

Via Christopher Blizzard, who found this by way of JavaScript guru John Ressig who did the JS port in the first place.

Apple’s iPhone — and the significantly more affordable, doesn’t-have-to-be-a-phone iPod Touch — are essentially pocket-sized, intelligent multi-touch controllers. Hooking them up to visual software as controllers simply requires some app on the phone to transmit data, and some way of dealing with that data on the computer side. We’ve already seen this a bit on Create Digital Motion, and we’ve been covering some of the specifics of parsing data with Pd (Pure Data), the open-source, tri-platform patching software, on Create Digital Music this week.

Here’s the basic setup:

On Your iPod/iPhone

You have two options of software to use on your iThing. (You’ll need to “jailbreak” your device, as these are not — and may never be, for all I know — approved Apple apps.)

1. mrmr by Eric Redlinger of Brooklyn (top right):  open-source, editable control screens (requires Mac-only software to edit). See our interview with Eric, including some examples with Quartz Composer.

2. akaRemote.app by Masayuki Akamatsu of Japan: not open-source, not editable, but comes with a set of useful control templates, and you can transmit data to the app. See our look at a recent release. Upcoming Mac-only visualist app 3L has its own special akaRemote-based bridge called i3L, which also runs on iPhone/iPod Touch; see our look at i3L with artificial eyes.

On Your Computer

While the iPhone and iPod Touch have Apple logos on them, all of these apps send OpenSoundControl data. That means any OSC-compatible software will work, which is gradually including more visual software, as well as modular apps like Quartz Composer, Max/MSP/Jitter, Pd/GEM, and vvvv. (I love saying that last one … vvvvvvvvvvvvvvvvvvv. Okay, moving on.)

Of all of these, Pd is the one solution that’s free, open source, and runs on any platform. That means it’s also a viable candidate for translating incoming OSC data to more broadly-compatible MIDI. (i3L has you covered, as it uses a MIDI bridge.)

We have not one but two sets of tutorials / example patches for working with Pd on Create Digital Music, using a patch like Cesare’s, pictured above:

Control Music and Visuals with iPhone/iPod, Free Via Pd

Tutorial: More iPhone/iPod Touch Control With Open-Source Pure Data

So, Is It Worth It?

I usually don’t ask that question, preferring instead to report on what other folks are doing. But it is always worth asking yourself — and it is an entirely personal question. I’m not totally convinced in the case of these devices that I’d want to buy one solely for VJing, but then, what makes this so cool is that it adds on additional functionality to a device. (Too bad Apple is being so uptight about third-party development, but at least there’s an SDK — and plenty of hackers ready to break Apple’s rules.)

My own preference remains squarely with tangible controllers and tactile feedback, especially as some of the advantages of multi-touch are diminished by the iPod/iPhone’s diminutive size. But I absolutely see the argument for using these. What do you think, dear readers?

iPhone development continues despite a lack of SDK. Masayuki Akamatsu has updated his native iPhone OSC client aka.iphone to 2.1, and includes plethora of screen/input configurations to suit your mobile performance needs. The update should suit typical VJ and Dj applications with its range of GUI sliders, knobs, button grids, etc.

If you have an iPhone or iPod Touch, be sure and check out aka.iphone. Very cool stuff. Yes, sorry, another Apple oriented update. Shoot me.

[Ed.: Note you will want Max/MSP/Jitter to work with this app, though I think it should work with Windows, too; I just lack a machine that can test it. Be sure to see also Masayuki's complete Max collection, which now covers everything from Wii remotes to 3D SpaceNavigator controllers, Apple Remotes, and even the motion sensor in the MacBook Pro. Completely brilliant. And needless to say, even if you don't want to hop on AT&T's mobile network, this gives you options with iPod Touch, too, even if you can't play Phase on that. -PK]

Mobile, touch control of visual apps? We’ve seen one way to do it; here’s an even more compatible MIDI approach. I think I’d stick to the iPod Touch, personally, just to conserve battery on a phone. But it’s interesting. -PK

Hot on the heels of Mrmr, ArtificialEyes (the Istanbul-based VJ collective) have released i3L MIDI Bridge for the iPhone, which builds on Masayuki Akamatsu’s aka.remote.app, giving those without Max/MSP skills a simple way to bridge their iPhone and audio/video performance software of choice.

i3L (pronounced “i thrill”) is Freeware, and was developed using Max/MSP from Cycling74 and is a support application for aka.remote by Masayuki Akamatsu i3L receives pre-defined UDP messages from aka.remote.app running on the iPhone, scales the values to MIDI, and allows you to configure the sending MIDI channel and control change message number. While this software was developed to work seamlessly with our Real-time 3D VJ software Thrill, you can use this program with any audio or video software which receives midi messages.

If you give i3L a try, let us know how it works out for you in the comments.

artificialeyes.tv
aka.objects by Masayuki Akamatsu — must-have stuff for Max/MSP/Jitter!

Ed.: Does anyone know if multi-touch gestures would be possible in DIY apps? Maybe in February when Apple shares the official SDK? -PK

Updated: Okay, maybe I’m flying off the handle on this one, after all. iLounge reported incompatibilities with the new iPod video output capability. They put it this way:

Without disclosing the change to customers, Apple has locked the TV Out feature of the iPod classic and video-capable iPod nano, preventing users from outputting iPod content to their TV sets as has been done in years past. Going to the Videos > Settings menu brings up a TV Out option that is now unresponsive when clicked, showing only the word “off.” When locked, video content will display on the iPod’s screen, but not on your TV or portable display accessory.

Apple locks TV Out in new iPods, breaks video add-ons [iLounge]

I’m going to save my “Apple’s iPod is closed” argument for another day, because I have to admit, I may be overreacting without the actual details. (For those of you who read my rant, hope you enjoyed it; now I’ll formally retract it until I know what the heck I’m actually talking about.)

Basically, here’s the problem:

There may not be any “authentication chip”, and this isn’t necessarily DRM. I was wrong; Anton and others are right: there’s no confirmation of an “authentication chip” or that this is intentional video DRM. If it is DRM, of course, it’s pretty stupid, as it won’t stop anyone from recording the video output of an iPod. It’s more likely that it’s just an incompatibility with existing products, but then, aside from cases and such, for the most part the iPod (and Apple TV) are pretty closed products to begin with. (I personally think that’s a problem, but that’s not DRM — it’s just choosing to have stricter control over the product.)

The cable is $50 because it has a power adapter. Okay, that makes more sense: one cable gives you video plus power. I’d still like to see a separate video cable, though, and I’m no huge fan of the iPod’s weird connector (of course, it’s there to keep the design thin).

There still are underlying problems, though, trying to use the iPod as a more flexible device. This just demonstrates that it’s really not designed that way. In some cases, I think that gives Apple more control over the consumer experience than they should have, and I really don’t think it’s healthy that the iPod dominates the market the way it does. At the same time, I also recognize my needs are different from the average consumer — but, hey, if you enjoyed being just an average consumer of technology, you wouldn’t be reading the CDMs, would you? Whether or not it has an impact on Apple’s larger business, I think we can safely enjoy the bleeding edge.

Anyone know of good Linux-based devices with video output capabilities? The GP2X is looking better than ever. S-Video out of a GP2X mixed with a computer could be ideal — and the GP2X appears to have better video format support than the iPod.

iLounge is reporting that the new iPods released wednesday require some sort of authentication/DRM chip for video output cables to work. Apparently, existing video output solutions from 3rd parties that work with the 5G/5.5G video iPod do not work on the Nano and the Classic (the iPod Touch is not currently shipping - but assumed not to work as well). This means you have to buy an Apple, or newer ‘compatible’ (authorized?) 3rd party video output cable. I’m uncertain if this is actually due to a change the circuitry, due to the addition of component video output, or a deliberate addition of DRM.

Apple Locks TV out in new iPods - breaks video add-ons - iLounge Read on for reports of what works and doesnt.

No doubt you are aware of Apples updated iPod line, with the new flagship iPod Touch bringing the iPhone’s touch screen interface to more affordable waters. However, what I find interesting is that every iPod now supports 640×480 video playback and output, and component video output via the new Universal Dock and AV Cables*.

While compressed video with component cables seem like conflicting ideas**, the iPods support fairly high bitrate h.264 and MP4 video. With the new iPod Classics sporting up to 160GB of storage, thats a serious VJ Clip library in your pocket. Add on something like Karl Klomps Dirty Video Mixer and you have an incredibly porable rig. And with two iPod Nanos you might very well have the worlds smallest VJ Mixing rig?

Karl Klomps Dirty & Cheap Non-Sync Video Mixer. Awesome.

While not the most versatile setup, it seems a rather tempting solution just for novelties sake. What seems more fun, two iPod Nanos and a tiny homebrew glitch video mixer in your pocket or a V4, two laptops and a camera, and maybe a triggering midi device. Hm.. The Component video also gives you access to the RGB (or possibly YUV) color information, allowing for some interesting mixing should you feel the urge to get creative. Speculation aside, Portable Media Players are looking quite intriguing for VJs lately. That iPod Touch is looking awfully tempting, especially with homebrew applications and OpenGL acceleration.

Are any Create Digital Motion readers currently using some sort of Personal Media Player or Video iPod for clip triggering in sets, or other similar creative solutions?

*sans shuffle, but, its never really counted now has it?
** Component video cables output a cleaner signal, while highly compressed video is not pristine. If you have highly compressed video, you probably aren’t worried about quality.

Video production ubersite Creative Cow has two great new After Effects video tutorials.

First up: Andrew Kramer’s “iPod Nano Light Streaks”. Using Trapcode Particular and a custom particle shape with healthy doses of hue adjustment and glow to produce an effect similar to the newest iPod ads. Adding some subtle animation to the custom particle would give this a more organic feel closer to the original.

Next Up: Aharon Rabinowitz explains Cycore’s Force Motion Blur. Cycore Effects comes bundled with AE 5.6 and 7.0, so if you don’t see it in your effects menu: crack out that install CD. Aharon demonstrates some great uses for this effect, but he brings up an important point for general After Effects use: If you apply an effect which doesn’t seem to do anything (or seems to turn off other effects), try applying it to an adjustment layer or precomp.

If you hunger for learning, Andrew Kramer’s site features a range of tutorials, and Aharon’s After Effects Podcast has some great ideas and workflow tips.

Stephen Schleicher has a great tutorial at Digital Media Net for
recreating those slick moving arrows in Apple's iPod Shuffle Ad.
Basically, you need Adobe After Effects and a working knowledge of
paths. See Stephen's site for more AE tutorial goodness.

Tutorial Part I

Tutorial Part II

Which brings up an interesting question — how many of you are doing
your own motion graphics / VJ work for your performances? I've been
getting into it gradually; expect some reports this summer.