The typical user likely has no idea that Flash is even supported in QuickTime. But VJ apps like ArKaos VJ and other applications (like Max/MSP/Jitter patches that use this support) rely on QuickTime to support Flash. Unfortunately, as of Leopard — or any Mac updated to QuickTime 7.3.1 — that support is gone, and will most likely never return.

What’s the reason? Security:

Multiple vulnerabilities exist in QuickTime’s Flash media handler, the most serious of which may lead to arbitrary code execution. With this update, the Flash media handler in QuickTime is disabled except for a limited number of existing QuickTime movies that are known to be safe. Credit to Tom Ferris of Adobe Secure Software Engineering Team (ASSET), Mike Price of McAfee Avert Labs, and security researchers Lionel d’Hauenens & Brian Mariani of Syseclabs for reporting this issue.

About the security content of QuickTime 7.3.1 (thanks to n0s10pm for the tip!)

In case this isn’t clear, this impacts only embedded Flash files within QuickTime, not Flash itself. But because some apps use QuickTime as a way of hooking in support for Flash animations, that’s a big deal — at least for someone. Windows users are impacted, too; this covers both platforms, though at least Windows users have the option of using Flash integration features available only on that OS.

Now, admittedly, the small niche pushing the limits of QuickTime and digital visuals is dwarfed by the potentially-enormous crowd who could be impacted by this vulnerability. But there’s clearly a philosophical problem here we’ve seen before: instead of fixing the vulnerability and providing a replacement, the entire idea is axed. Necessary? Perhaps, but I would love to see Adobe address ways of more easily integrating Flash technology with other applications. As the Flash/Flex platform grows, the potential here could be wide enough to justify the effort, especially as applications (not just VJ apps) add rich client features.

And in the meantime, if you use a tool that uses this support, you may want to hunt down an old installer or hold off upgrading. (Reverting to older versions is, unfortunately, really hard — you have to completely uninstall QuickTime, and you’ll only have luck if you’re on Tiger or earlier as far as I know.)

Quartz Composer, first introduced in OS X 10.4 and rooted in the underground visual app PixelShox, is a hidden gem in Mac OS X. This free visual tool makes some sophisticated video and 3D magic possible without coding. While promising, the version in 10.4 had some significant shortcomings. We turn to Keith Lang of developer plasq, creators of Skitch and ComicLife, to give us an inside look. Keith is not a programmer — so he’s all the more eager to go the visual patching route — but he has had his hands on QC for some time in Leopard dev builds. And now, he’s free to tell us all he knows. -PK

Mac OS X 10.5 “Leopard” is here, and the initial hoopla about the reflective dock has died down. Now that we’ve all passed through the Seven Stages of Acceptance, we have an opportunity to take this brief slither of quiet to delve into the subtler changes within the big new cat. Those changes include improvements made to Quartz Composer, the quaint and oft-misunderstood visual processing environment bundled with the Developer Tools.

What You See: The Interface

For the uninitiated, Quartz Composer is an ‘environment’ tool which lets you patch together various elements in order to build screensavers, performance environments, and animated windows for use within an application. If you’ve ever looked at an Apple screensaver, played with the new iChat Effects or watched photos float and twirl on an Apple TV, then you’re familiar with what Quartz Composer can do. Its peers are tools like Pd (Pure Data), Processing, and Max/MSP/Jitter, with Quartz Composer heavily geared towards motion animation of processed video and images. Input can be everything ranging from your iSight to XML to audio to a Wiimote, with output being a window on 3d space containing all the sprites, effects and movement you care for. Because Quartz Composer is a real-time system, you can plug stuff together whilst watching and tweaking. The underlying engine is very efficient - and with a little work you can create teh cool, without ever needing to type teh code.

Create Digital Music got an interesting comment about allegedly, what really happened to Java in Leopard. Short version: nearly all of the development team is gone; the one Java 6 person left and many others were moved to, of all things, iCal. Java 5 is likely to be all you get on Leopard, and still without improvements to JavaSound or QuickTime for Java, which have left sound and video support pretty weak. (QuickTime for Java is an even worse choice on Windows, though at least on Windows and Linux JavaSound runs better.)

Caution: this could be no more than a rumor. There seems to still be a Java development team at Apple, so at the very least, it sounds exaggerated. However, it’s also clear that Apple’s commitment to developing Java on the Mac is tenuous to say the least.

For Processing fans, though, there’s various good news. First, Apple has updated Java 5, the version of Java targeted by Processing currently. Secondly, while JavaSound still lags behind major quality improvements made on Mac and Linux, if Apple really is abandoning Java on the Mac, it could mean the finger pointing is over and finally Sun could develop Java on the Mac platform, bringing all three desktop OSes in sync (sorry, Solaris, that’s Linux, Mac, and Windows for nearly all of us). Another interesting tidbit: JavaFX, if Sun is to be believed, could be more than a scripting language, it could provide better multimedia support long lacking in Java. I imagine that could be incorporated into Processing. See the CDM story for more details — taking them with a grain of salt, as none of this has been officially confirmed.

Rumor: Mac Java’s Demise is Real, and Why That Could Be Good News for Multimedia

Believe it or not, people making art with webcams don’t rate very highly on the priority list for big computer companies. (Who would have thought?) On the PC, at least, there’s a thriving market for webcams for video chat, since so few PCs have built-in cameras. Meanwhile, on the Mac, Apple has absolutely zero interest in you using any webcams other than those built into their machines, or, if you’re lucky, one of the FireWire iSights Apple made before Apple discontinued them. (Given the high failure rate I’ve seen on the iSights, that assumes you’re lucky enough not only to have found one, but to have it still working.) Ditto, naturally, third-party manufacturers, since there’s unlikely to be any significant market for their wares — and they’re busy navigating the morass of driver development complexity on PCs.

Long story short: the Creative Labs Live! Optia I raved about in the fall is one of the few choices you’ve got that doesn’t require drivers. It’s USB video class-compliant, though unlike other USB classes, it’s not entirely clear that that’s all that meaningful.

But, for several glorious months, through last week, I was able to keep my Live Optia working perfectly with Processing (and thus QuickTime for Java) and QuickTime (via tools like Jitter). Until today, that is. Now I’ve got two of them, five Macs to test, and — nada. On 10.4.10 / QT 7.2 and 10.4.8 / QT 7.1.3 and 7.2, I get either a black screen or (in QuickTime video capture) garbled video. It looks like the sequence grabber isn’t properly setting the resolution, so pixels are being dumped arbitrarily from the camera … I suspect the other errors I’m seeing are also related. USB video class support is relatively new; it only hit iChat in 10.4.9 and may have reached the OS at the same time — I would know for sure, except documentation from Apple is scant.

I suspect some misbehaved QuickTime update, though I find it especially odd that it fails on multiple machines (all Intel — iMac, MacBook, MacBook Pro, and Mac mini) with different versions. I’ve tried reinstalling QT, zapping NVRAM (formerly PRAM), the lot. For once, I can’t blame QuickTime for Java, because everything else is broken, too.

Webcams working some of the time under unpredictable circumstances don’t inspire confidence. Suggestions, anyone? Any idea why this is happening? Anyone got a rock-solid solution for Mac webcams that doesn’t spontaneously cease functioning?

Incidentally, Windows isn’t much better; weird driver bugs there can cause fabulous results like an echo-cancellation driver knocking out USB MIDI devices, driver-related blue screens of death, and other goodies.

Maybe I should just start making my own cameras and writing my own drivers. Yeah, that’s it.

We already knew that Leopard would be a major new OS release for Apple in the visual sense with Core Animation, much as Tiger was with Quartz Composer, Core Video, and Core Image. We should know a lot more at Apple’s developer conference in June. There are some additional details in Apple’s latest WWDC announcement, under “Leopard Innovations”:

1. Core Animation to add elegant animations to your application interface.
2. iChat Theater to incorporate application content in iChat.
3. QuickLook to provide system wide previews.

Core Animation is old news, but could this mean you could add iChat streaming to an app? And system-wide previews are also badly-needed.

Surprise! The one application with which I’ve had the most success on my Vista turns out to be Max/MSP/Jitter, the niche, Mac-first multimedia application built by a small handful of developers. Without so much as a single update, Jitter is running perfectly on Vista, running on an NVIDIA GeForce Go 7600 card on an Alienware m5550. (Next stop: upgrading my home-built Shuttle XPC SFF machine.)

The good news about this, especially in light of the generally mixed results in gaming, is that Jitter is a deeply OpenGL-dependent application. And for Windows users who have tired of glitches and stutters when showing OpenGL in a window on XP, you’ll love Vista. Thanks to GPU-bound graphics, windows animate smoothly, as on OS X. All I do notice is a slight delay when switching to full-screen, but these not-yet-released NVIDIA mobile drivers have been exhibiting little glitches each time the UAC (User Account Control) dialog appears. Otherwise, I’ve had good experience with this, and tellingly switching back from full-screen mode to pointless GPU eye candy like Flip 3D doesn’t cause a single problem. Vista could wind up being a great OS for running your home-built Max patches after all.

One major caveat that should keep Jitter users from rushing to upgrade, though: PACE copy protection isn’t officially supported yet, meaning you may have issues authorizing your copy of Max on Vista. I expect an update will come soon based on what I’ve heard from Cycling ‘74 and PACE. So, stay tuned; I’ll be sure to post more detailed Jitter results and information on the update once it becomes available.

Anyone else using Vista aside from me? More reports? I’m too scared to try Processing + Java; different Processing versions combined with different Java runtimes have been enough of a moving target as-is.

Vista for visualists is a problematic proposition. Typically, musicians have the toughest upgrade curve with any OS, but there’s a major incentive for musicians with Vista, as reported on CDM: the promise of significantly improved audio performance and stability. Right now, there isn’t enough software or hardware support for most musicians, though I expect that picture will start to change even within the next few weeks.

For visualists, though, we have two problems:

1. The only really major improvement (beyond what’s in Vista itself) is the new 3D platform, DirectX 10. That could be huge for advanced visualists willing to be platform-specific — but with the development tools only just now available and a lot of Microsoft’s own development initiatives (like XNA) still tied to DirectX 9, even programmers are unlikely to do anything with DX10 in the near future, let alone end users. (Cheap cards will be out soon, but more on that in a separate story.

2. So far, the big hole in Vista is graphics support. So, not only is the step forward (DX10) not quite ready for most users, in the meantime OpenGL and even DirectX 9 will see a big step backward for the short term.

Linux is looking better and better as a fully open-source, multimedia-optimized operating system for visualists. Adobe has announced the availability of Flash Player for Linux. The bad news is, it’s x86 only, and as expected, there may be some issues. But the good news is, it’s here, and early reports indicate it works. Even as a regular Mac and Windows user, I can see building a cheap PC box for an installation and saving the cash I’d normally have to spend on a Windows license. Download Squad has some good commentary and links. If you’re on Linux, just go grab the player the normal way, and you’ll get a Linux option (neat!)

Flash Player 9 for Linux Final Released

What else does this mean? It means you can go fully free + open-source with your next VJ set, taking advantage of the ActionScript 3-based project Onyx pictured here:

Onyx VJ

I’m converting a PC laptop here to a Linux-only box. One of the first things I want to test: video4linux Flash support, if Adobe delivered that as promised. If you’re trying a mixed-OS environment on the Mac, be sure to catch the other gem from Download Squad, a how-to for MacFUSE, which gives you Linux volume access on the Mac.

pure:dyne is a new free, open source, Linux-based, bootable, low-latency, high-performance operating system with Pure Data (Pd) as its main emphasis. Pop a CD, USB key, or bootable hard drive into your PC or Intel Mac, and you’re ready to go.

pure:dyne, the Art + Music Performance OS for PCs and Intel Macs [Create Digital Music]
pure:dyne Wiki

Now, needless to say, a big appeal of this isn’t just tuning the OS to your needs — it’s taking performance gigs wherever you go, ready to go, without having to worry about OS crashes. It’s installing a pre-built installation on a home-built Linux computer or Mac mini. It’s having a backup when your machine dies.

pure:dyne is completely tuned for Pd, the open-source cousin of Max/MSP. (Pd was created by Max’s creator, Miller Puckette, and Max and Pd have shared code and exchanged ideas since the very beginning.) Pd comes with an awesome lineup of Pd extras: PDP, PiDiP, Gem, GridFlow, RRadical, PixelTango — all the additional libraries that normally take some time and dependency-managing to install (and some of which just don’t work or don’t work as well on Windows or even Mac). So, if you’re ready for some free patching of custom visuals, you’re already happy.

But this got me thinking. Processing runs under Linux, and one challenge is often tuning a distro with the right drivers and settings (think webcam support, for one) and different Java versions. It’d be a no-brainer, if Processing is to be as successful as it could be, to tie it to a perfectly-tweaked, bootable distro. Flash, now with the robust Flash Player 9 for Linux, which also promises video4linux webcam support for live input, could do the same. (In fact, for my own purposes, a custom Pd+Flash+Processing distro would be heaven for live visualist work.) And that’s just the beginning.

This also makes a nice bridge between commercial software (the comfort of Adobe Creative Suite for assembling visuals, for instance) and free software (a perfectly-tuned, custom performance and installation environment).

So, what do you think? Who’s with me? And what would you want out of such an OS — or do you know of some places to get started? (One obvious starting point would be dyne:OS, the core on which pure:dyne is built.)