Joshue Ott/superDraw +Ezekiel Honig live at monkeytown from superdraw on Vimeo.

I’m very excited about the music lineup we have planned for this Friday in New York at the CDM-curated evening of live audio and visuals – but the visual lineup should be a big draw, too. If you’re in New York, come say hi (and if not, hope to have more details on these projects for the rest of the planet soon):

• Joshue Ott creates live visuals with his homemade superDraw generative illustration tool
• Paris (Voltage Controlled) and Don Miller (No Carrier) create glitchy, lo-fi visuals from custom-created 8-bit visual software on Nintendo and Commodore systems
• vade and Mary Ann Benedetto will visualize and reinterpret geeky things (possibly the Linux kernel, data packets, or both) using custom code and Quartz Composer stuff — we should even see a free release of some of those tools in time for the gig, so stay tuned to CDM
• Bill Jones creates live cinematic worlds inspired by sci-fi noir

Where: The Hotel Pennsylvania, New York City (map); head to the main door, on your left is the entrance to Penn Pavilion and you should see a table there.

When: Friday, July 18 2008 – performances run 11pm – 2am

Cost: US$10 at the door. First come, first served. (free if you have a conference badge)

We Are Hacks: Music and Visual Performance at HOPE, NYC – Preview

http://www.thelasthope.org/

Facebook event page (RSVP if you’re coming! Also on Going.com)

Above: one of my favorite videos from superDraw (Processing-based) by Joshue Ott above, though it’s even better to see it in person with the live drawing capabilities. Below: all-custom 8-bit-style software generates visuals, via Paris.

Function Field System - PureData/GEM from Paris/VoltageControlled on Vimeo.

CDMotion contributor vade has posted the first release of his v002 Screen Capture tool, which allows video from the screen (including video, 3D — anything output to OpenGL) to be routed between applications. It all happens on the GPU, which means it’s very, very fast. In vade’s words:

v002 Screen Capture allows you to capture your desktop, or a portion of it to a texture and further process it. This can be used to bring in other applications output or windows as a source input to VDMX or other Quartz Composer compatible patch hosts.

Screen Capture is fully GPU accelerated, and therefore is very fast.

Sample Processing, 3L, Modul8, Jitter, GEM, or any application, and mix them in VDMX, or your Quartz Composer patch host of choice.

Right now, the release is Quartz Composer and Mac-only. (Quartz Composer plug-in support means it’ll also drop nicely into software like VDMX.) But there’s an open call to port this to other environments (Pd, Max/MSP/Jitter, Processing, and such). It may even be possible to replicate the basic technique on another operating system, though the implementation would have to be reconsidered.

We’d love some feedback, so have at it! Especially interested in Processing support; see the thread on the Processing forums.

v002 Screen Capture Quartz Composer plug-in download

Apple’s iPhone — and the significantly more affordable, doesn’t-have-to-be-a-phone iPod Touch — are essentially pocket-sized, intelligent multi-touch controllers. Hooking them up to visual software as controllers simply requires some app on the phone to transmit data, and some way of dealing with that data on the computer side. We’ve already seen this a bit on Create Digital Motion, and we’ve been covering some of the specifics of parsing data with Pd (Pure Data), the open-source, tri-platform patching software, on Create Digital Music this week.

Here’s the basic setup:

On Your iPod/iPhone

You have two options of software to use on your iThing. (You’ll need to “jailbreak” your device, as these are not — and may never be, for all I know — approved Apple apps.)

1. mrmr by Eric Redlinger of Brooklyn (top right):  open-source, editable control screens (requires Mac-only software to edit). See our interview with Eric, including some examples with Quartz Composer.

2. akaRemote.app by Masayuki Akamatsu of Japan: not open-source, not editable, but comes with a set of useful control templates, and you can transmit data to the app. See our look at a recent release. Upcoming Mac-only visualist app 3L has its own special akaRemote-based bridge called i3L, which also runs on iPhone/iPod Touch; see our look at i3L with artificial eyes.

On Your Computer

While the iPhone and iPod Touch have Apple logos on them, all of these apps send OpenSoundControl data. That means any OSC-compatible software will work, which is gradually including more visual software, as well as modular apps like Quartz Composer, Max/MSP/Jitter, Pd/GEM, and vvvv. (I love saying that last one … vvvvvvvvvvvvvvvvvvv. Okay, moving on.)

Of all of these, Pd is the one solution that’s free, open source, and runs on any platform. That means it’s also a viable candidate for translating incoming OSC data to more broadly-compatible MIDI. (i3L has you covered, as it uses a MIDI bridge.)

We have not one but two sets of tutorials / example patches for working with Pd on Create Digital Music, using a patch like Cesare’s, pictured above:

Control Music and Visuals with iPhone/iPod, Free Via Pd

Tutorial: More iPhone/iPod Touch Control With Open-Source Pure Data

So, Is It Worth It?

I usually don’t ask that question, preferring instead to report on what other folks are doing. But it is always worth asking yourself — and it is an entirely personal question. I’m not totally convinced in the case of these devices that I’d want to buy one solely for VJing, but then, what makes this so cool is that it adds on additional functionality to a device. (Too bad Apple is being so uptight about third-party development, but at least there’s an SDK — and plenty of hackers ready to break Apple’s rules.)

My own preference remains squarely with tangible controllers and tactile feedback, especially as some of the advantages of multi-touch are diminished by the iPod/iPhone’s diminutive size. But I absolutely see the argument for using these. What do you think, dear readers?

After twenty years or so of music software, you’ll often find that what you need to do is creatively satisfied by what’s available in pre-built tools. But video is often another matter: even if a VJ tool does what you need some of the time, there are times when you need something that doesn’t exist (or even something simpler, to perform a specific task). You might not be ready to invest hundreds of dollars in Max/MSP/Jitter to do that, of course. Enter Pure Data, Max’s free, open-source cousin. Like Max, it lets you quickly build custom tools by visually “patching” objects on-screen — ideal for modular video tools. But unlike Max, it costs nothing, it runs on Mac, Windows, and Linux, and because of the rich Pd community and a good relationship with the Max community, it has some tricks of its own.

Beau, aka DJCypod, writes to share his excellent video tutorial on building a simple 2-way mixer in Pd. Beau, if you ever need a second career, I think you could do relaxation recordings; I felt my blood pressure dropping. Next time I have students getting nervous about Pd or patching, I’m sending them to this video. (Well, and also because it’s really easy to follow.)

Totally beginner-friendly, so newbies will get rolling fast. And as a long-time Max user, it served as a nice introduction for me, too.

Pure Data Video Mixer: YouTube, Internet Archive

You’ll also need the Pd Extended Release; see our previous story

I’ll be building some Pd software mixers this month, so I’ll be sure to report back! (Any requests, or tips from those of you who know Pd better than I do?) Now, if only I could easily integrate Processing with Pd, as you can with Max. (At least Pd does have Java support of its own.)

Open Source software can be hard to approach sometimes, especially DIY programming environments, so there is some good news for those wanting to get started with two leading open source packages for performance.

Pure Data - the open source dataflow/patching environment now has a shared playlist on youtube. If you’ve been interested in Pure Data but hadn’t the slightest clue what it is or how to use it, you should check out the Pure Data Videopedia. The videopedia contains performance clips, tutorials and examples made with PD and some video related plugins.

Not into datalflow/graphical programming environments? Get your hands dirty with Processing. Check out the recently posted tutorials by user b2kn : “Coding for visual performance” workshop. You can find the accompanying processing sketches here. Good stuff!

Pd, aka Pure Data, is the free and open-source cousin of Max/MSP/Jitter. It’s powerful — even sometimes having technical advantages over Max — but has suffered from complex installation and dependencies, poor documentation, and an unpolished interface. Enter Pd-extended, a distribution that fills in those gaps. Pd-extended’s maintainer Hans-Christoph writes up what this is all about in an introduction on Create Digital Music, friendly even if you’re new to the Pd world.

Pd, Max’s Free Cousin, Gets Polish and Ease in Extended Build

Visualists should be especially interested in this latest release, because it offers much-improved out-of-the-box support for custom-patched 3D and video — especially if you’re on a Mac, for the PiDiP (though there are Windows and Linux improvements, too, and GEM works even with Windows).

Mac OS improvements:

• Image and video-processing PDP/PiDiP work out-of-box
• Anti-aliasing of boxes and lines in the interface
• New, purty icon

Linux:

• A .deb package for Debian and Ubuntu, with GNOME menu support. (`Bout time! Wonder if this means we’ll see it in the big Ubuntu repositories?)
• New icon

All platforms:

• GEM, the quasi-Jitter-like 3D and pixel library, has working shader support. Ed.: Truly outrageous.
• New libraries: mapping, msd, mrpeach net/OSC, flib
• [comport] is robust on all platforms (can you say Arduino?)
• Font-face and -font-weight command line options
• New font and layout is the exact same size on all platforms to the pixel. (previously you’d see some serious cross-platform glitching)

Pd Extended Release

And lest you have a bad taste in your mouth from the fugly older releases of Pd, Anton (vade) sends along the picture at the top of this story, showing the new UI from the Pd 0.40 dailies running on Mac OS X. Anton is also working on porting some of his brilliant visual patching from Max to Pd — and he’s living proof that even a Max die-hard can find at least some use with Pd, too. (The two environments are really, really close — sometimes confusion switching between them is because they’re so close, the differences can be confusing.)

With the advent of consumer-level HD camcorders, Blu-Ray and HD DVD players, and HD-format disc burning, HD content creation is becoming ubiquitous. However, for realtime visualists looking to mix HD sources live, there hasn’t been a real HD mixing solution. Want to mix that HD footage on your PC with that awesome high-res project in VJ software like Modul8? No can do, buddy!

At least, that had been the case. With two Blackmagic Design Intensity cards and Blackmagic’s On Air software, you can mix two streams of HD as a simple AB mixer. With one Intensity in your machine, you can use a second laptop or source to feed your main VJ / live visual app of choice with true, uncompressed HD input.

We tested live, HD mixing for color performance, quality, and ease-of-use with some powerful, DIY visual software for live visuals. With the ability to mix in live HD inputs, live HD cameras, live computer inputs at full HD resolution, and more, the potential for live visuals is clear.

In the immortal words of Burt Bacharach:

Beware of The Blob, it creeps
And leaps and glides and slides
Across the floor
Right through the door
And all around the wall
A splotch, a blotch
Be careful of The Blob

It’s true, Burt. Tracking multiple blobs in an image for motion tracking, in order to analyze a video source to track points of movement within that image, is hard. The open source, multimedia processing Pure Data library Gridflow does a lovely job in a new patch by Mathieu Bouchard. Found via the excellent PD-Announce list, which manages to be useful but not overwhelming. Mathieu writes:

It’s actually my own implementation: it’s not been ported from anywhere else. Each cross is centered on the centroid (1st order moment) of its corresponding region (”blob”), and the size of the cross is the square root of the area of the region.

I will make another version which will use rotated rectangular crosses. The two lines of each cross will represent the two radiuses of the standard deviation of the region, and the angle will represent the angle of the standard deviation. I will use [#moment_polar], an abstraction based on [lti.Jacobi].

Mathieu explains how to get it (ain’t open source great?):

If you get your GridFlow from the CVS, just update and you’ll have [#labeling].

(It’s called like that because it “labels” each region of the picture with a different number, so it creates something called “a labeling”).

You’ll find lots of Pure Data goodness at Mathieu’s site. See the full image of the patch.

Mostly, I’m going to use this as yet another excuse to go grab the pure:dyne Linux distro. GridFlow and Pd are already on there, so you need to know very little about setting up Linux and installing them. pure:dyne should be Intel Mac ready any day now, and it’ll boot from a key, a hard drive, or a disc, so it’s very easy to just boot it up and give this stuff a go. I’m dying to run it on my MacBook, so I’ll post here once that version becomes available. (Note, Mathieu observes that of course this is too new to be on pure:dyne yet … but that’s okay with me; it still demonstrates the utility of Gridflow, and there’s nothing stopping you from using pure:dyne as a starting point and switching to CVS if you can handle a little bleeding-edge code.)

pure:dyne is a new free, open source, Linux-based, bootable, low-latency, high-performance operating system with Pure Data (Pd) as its main emphasis. Pop a CD, USB key, or bootable hard drive into your PC or Intel Mac, and you’re ready to go.

pure:dyne, the Art + Music Performance OS for PCs and Intel Macs [Create Digital Music]
pure:dyne Wiki

Now, needless to say, a big appeal of this isn’t just tuning the OS to your needs — it’s taking performance gigs wherever you go, ready to go, without having to worry about OS crashes. It’s installing a pre-built installation on a home-built Linux computer or Mac mini. It’s having a backup when your machine dies.

pure:dyne is completely tuned for Pd, the open-source cousin of Max/MSP. (Pd was created by Max’s creator, Miller Puckette, and Max and Pd have shared code and exchanged ideas since the very beginning.) Pd comes with an awesome lineup of Pd extras: PDP, PiDiP, Gem, GridFlow, RRadical, PixelTango — all the additional libraries that normally take some time and dependency-managing to install (and some of which just don’t work or don’t work as well on Windows or even Mac). So, if you’re ready for some free patching of custom visuals, you’re already happy.

But this got me thinking. Processing runs under Linux, and one challenge is often tuning a distro with the right drivers and settings (think webcam support, for one) and different Java versions. It’d be a no-brainer, if Processing is to be as successful as it could be, to tie it to a perfectly-tweaked, bootable distro. Flash, now with the robust Flash Player 9 for Linux, which also promises video4linux webcam support for live input, could do the same. (In fact, for my own purposes, a custom Pd+Flash+Processing distro would be heaven for live visualist work.) And that’s just the beginning.

This also makes a nice bridge between commercial software (the comfort of Adobe Creative Suite for assembling visuals, for instance) and free software (a perfectly-tuned, custom performance and installation environment).

So, what do you think? Who’s with me? And what would you want out of such an OS — or do you know of some places to get started? (One obvious starting point would be dyne:OS, the core on which pure:dyne is built.)