Using video in Processing is, sadly, really painful. You can do absolutely wonderful things once it’s working — pixel-by-pixel manipulations that are hard to do elsewhere, and easily-coded OpenGL manipulations that should help generate powerful eye candy. But the list of issues runs something like this:

• Windows doesn’t support capture without the addition of the buggy WinVDIG, which often doesn’t work properly.
• Linux doesn’t support anything.
• Capture is slow on Mac, and sometimes doesn’t work.
• QuickTime updates regularly hose the whole setup.
• Important QuickTime features aren’t supported.
• Playback is slow.
• Playback often crashes.
• Things just don’t work, and you don’t know why.

Now, granted, for simple sketches and experimentation, the library often will work. Or at least, it works except when it doesn’t — and you can read about how often it doesn’t in the reference.

Fortunately, I think the situation could be — and soon will be — very, very different.

CDMotion contributor vade has posted the first release of his v002 Screen Capture tool, which allows video from the screen (including video, 3D — anything output to OpenGL) to be routed between applications. It all happens on the GPU, which means it’s very, very fast. In vade’s words:

v002 Screen Capture allows you to capture your desktop, or a portion of it to a texture and further process it. This can be used to bring in other applications output or windows as a source input to VDMX or other Quartz Composer compatible patch hosts.

Screen Capture is fully GPU accelerated, and therefore is very fast.

Sample Processing, 3L, Modul8, Jitter, GEM, or any application, and mix them in VDMX, or your Quartz Composer patch host of choice.

Right now, the release is Quartz Composer and Mac-only. (Quartz Composer plug-in support means it’ll also drop nicely into software like VDMX.) But there’s an open call to port this to other environments (Pd, Max/MSP/Jitter, Processing, and such). It may even be possible to replicate the basic technique on another operating system, though the implementation would have to be reconsidered.

We’d love some feedback, so have at it! Especially interested in Processing support; see the thread on the Processing forums.

v002 Screen Capture Quartz Composer plug-in download

Believe it or not, people making art with webcams don’t rate very highly on the priority list for big computer companies. (Who would have thought?) On the PC, at least, there’s a thriving market for webcams for video chat, since so few PCs have built-in cameras. Meanwhile, on the Mac, Apple has absolutely zero interest in you using any webcams other than those built into their machines, or, if you’re lucky, one of the FireWire iSights Apple made before Apple discontinued them. (Given the high failure rate I’ve seen on the iSights, that assumes you’re lucky enough not only to have found one, but to have it still working.) Ditto, naturally, third-party manufacturers, since there’s unlikely to be any significant market for their wares — and they’re busy navigating the morass of driver development complexity on PCs.

Long story short: the Creative Labs Live! Optia I raved about in the fall is one of the few choices you’ve got that doesn’t require drivers. It’s USB video class-compliant, though unlike other USB classes, it’s not entirely clear that that’s all that meaningful.

But, for several glorious months, through last week, I was able to keep my Live Optia working perfectly with Processing (and thus QuickTime for Java) and QuickTime (via tools like Jitter). Until today, that is. Now I’ve got two of them, five Macs to test, and — nada. On 10.4.10 / QT 7.2 and 10.4.8 / QT 7.1.3 and 7.2, I get either a black screen or (in QuickTime video capture) garbled video. It looks like the sequence grabber isn’t properly setting the resolution, so pixels are being dumped arbitrarily from the camera … I suspect the other errors I’m seeing are also related. USB video class support is relatively new; it only hit iChat in 10.4.9 and may have reached the OS at the same time — I would know for sure, except documentation from Apple is scant.

I suspect some misbehaved QuickTime update, though I find it especially odd that it fails on multiple machines (all Intel — iMac, MacBook, MacBook Pro, and Mac mini) with different versions. I’ve tried reinstalling QT, zapping NVRAM (formerly PRAM), the lot. For once, I can’t blame QuickTime for Java, because everything else is broken, too.

Webcams working some of the time under unpredictable circumstances don’t inspire confidence. Suggestions, anyone? Any idea why this is happening? Anyone got a rock-solid solution for Mac webcams that doesn’t spontaneously cease functioning?

Incidentally, Windows isn’t much better; weird driver bugs there can cause fabulous results like an echo-cancellation driver knocking out USB MIDI devices, driver-related blue screens of death, and other goodies.

Maybe I should just start making my own cameras and writing my own drivers. Yeah, that’s it.