The typical user likely has no idea that Flash is even supported in QuickTime. But VJ apps like ArKaos VJ and other applications (like Max/MSP/Jitter patches that use this support) rely on QuickTime to support Flash. Unfortunately, as of Leopard — or any Mac updated to QuickTime 7.3.1 — that support is gone, and will most likely never return.

What’s the reason? Security:

Multiple vulnerabilities exist in QuickTime’s Flash media handler, the most serious of which may lead to arbitrary code execution. With this update, the Flash media handler in QuickTime is disabled except for a limited number of existing QuickTime movies that are known to be safe. Credit to Tom Ferris of Adobe Secure Software Engineering Team (ASSET), Mike Price of McAfee Avert Labs, and security researchers Lionel d’Hauenens & Brian Mariani of Syseclabs for reporting this issue.

About the security content of QuickTime 7.3.1 (thanks to n0s10pm for the tip!)

In case this isn’t clear, this impacts only embedded Flash files within QuickTime, not Flash itself. But because some apps use QuickTime as a way of hooking in support for Flash animations, that’s a big deal — at least for someone. Windows users are impacted, too; this covers both platforms, though at least Windows users have the option of using Flash integration features available only on that OS.

Now, admittedly, the small niche pushing the limits of QuickTime and digital visuals is dwarfed by the potentially-enormous crowd who could be impacted by this vulnerability. But there’s clearly a philosophical problem here we’ve seen before: instead of fixing the vulnerability and providing a replacement, the entire idea is axed. Necessary? Perhaps, but I would love to see Adobe address ways of more easily integrating Flash technology with other applications. As the Flash/Flex platform grows, the potential here could be wide enough to justify the effort, especially as applications (not just VJ apps) add rich client features.

And in the meantime, if you use a tool that uses this support, you may want to hunt down an old installer or hold off upgrading. (Reverting to older versions is, unfortunately, really hard — you have to completely uninstall QuickTime, and you’ll only have luck if you’re on Tiger or earlier as far as I know.)

If you’ve been intrigued by all this talk of Quartz Composer, the free visual creation software that ships with Mac OS X 10.4 and 10.5, here’s your chance to actually learn how to do cool stuff with it. Our friend Momo walks us through a basic tutorial on simple 3D and audio processing, which you could easily apply to more complex ideas. With QC support in the upcoming VDMX5, you could drop this into a VJ set with traditional clips, as well. We’ve got step-by-step instructions, plus a video. Let us know if you create anything wild with this as its basis.

Quartz Composer: Lighting 3D Cubes and Moving them with Audio Input from momo_the_monster on Vimeo.

In this Quartz Composer tutorial, We’re going to make a 3D cube that responds to our voice.

First we start up Quartz Composer. The icon will be different depending on whether you’re running OS X 10.5 or 10.4.

From the File menu, choose New Blank (or simply ‘New’ in Tiger).

Quartz Composer gurus have had to face challenges bringing in network data: the problem is, to keep Quartz Compositions secure, Apple has largely crippled networking features. Celso Martinho has been hacking QC to make networking work in Leopard, and has a functional solution. He wrote us to tip us off on a detailed post at his blog.

First, if you’re still on Tiger, good news:

I needed a way to get data from the Network in the form of events that I could reuse in a quartz composition. So our resident mac programmer coded this custom made patch based on sparse non official documentation found on the internet. And it worked great. We have about 5 plasma screens with mac minis over at work running it for months, no problems whatsoever.

But while Leopard finally offered an official means of making your own patches (that’s what the rest of the patching world calls “objects”, Max/MSP, vvvv, and Pd users), Leopard also breaks their custom patch. Solution?

Then I found 2 patches in the new “Network” category: Network Broadcaster and Network Receiver. They are meant to connect several qtz compositions across the network and exchange messages between them. But maybe I can use them for something else…

I wrote a quartz composition to broadcast messages using UDP and multicast and started debugging and I discovered that the packets are really simple non-crippled text messages, four bytes per character iso-latin encoded chunks.

If you’re doing heavy-duty networking, I’d still investigate other alternatives to make sure Quartz Composer is your best choice. Processing and Max/MSP/Jitter both make short work of UDP send/receive, thanks to Java’s natural abilities there, as do objects in vvvv, Pd, and the like. Even Flash has some data features, with a little work. On the other hand, QC has some natural tricks of its own, and for multi-machine setups, the combination of this hack with QC’s new multi-computer features is very sweet indeed.

Full details, plus a PHP script that does the dirty work, here:

Leopard’s Quartz Composer and Network events [Celso Martinho]

Quartz Composer, first introduced in OS X 10.4 and rooted in the underground visual app PixelShox, is a hidden gem in Mac OS X. This free visual tool makes some sophisticated video and 3D magic possible without coding. While promising, the version in 10.4 had some significant shortcomings. We turn to Keith Lang of developer plasq, creators of Skitch and ComicLife, to give us an inside look. Keith is not a programmer — so he’s all the more eager to go the visual patching route — but he has had his hands on QC for some time in Leopard dev builds. And now, he’s free to tell us all he knows. -PK

Mac OS X 10.5 “Leopard” is here, and the initial hoopla about the reflective dock has died down. Now that we’ve all passed through the Seven Stages of Acceptance, we have an opportunity to take this brief slither of quiet to delve into the subtler changes within the big new cat. Those changes include improvements made to Quartz Composer, the quaint and oft-misunderstood visual processing environment bundled with the Developer Tools.

What You See: The Interface

For the uninitiated, Quartz Composer is an ‘environment’ tool which lets you patch together various elements in order to build screensavers, performance environments, and animated windows for use within an application. If you’ve ever looked at an Apple screensaver, played with the new iChat Effects or watched photos float and twirl on an Apple TV, then you’re familiar with what Quartz Composer can do. Its peers are tools like Pd (Pure Data), Processing, and Max/MSP/Jitter, with Quartz Composer heavily geared towards motion animation of processed video and images. Input can be everything ranging from your iSight to XML to audio to a Wiimote, with output being a window on 3d space containing all the sprites, effects and movement you care for. Because Quartz Composer is a real-time system, you can plug stuff together whilst watching and tweaking. The underlying engine is very efficient - and with a little work you can create teh cool, without ever needing to type teh code.

Via comments, Eelke (whose system is shown above) alerts us that previous video output problems running Windows in Boot Camp appear to be solved in Leopard — at least using the MacBook Pro. (Phew — get all that?)

See the ongoing thread on VJforums. Eelke says:

My configuration:
- MacBookPro 2,16, 2GB ram, with ATI X1600
- Upgraded to Leopard (OS X 10.5)
- Used the same Leopard DVD to update all drivers in bootcamp / XP
- Rebooted with DVI>s video dongle plugged-in, s-video out attached to small reference monitor

After the initial booting XP logo which we had all the time, I suddely saw my desktop cloned on the small screen. All I had to do is go to desktop>preferences to extend windows to the secondary monitor, and it worked.
After that I used ATI Catalyst control center to set the screen (now recognized as a TV!) to run in overscan, no problem at all, even during live rendering (with visualjockey).

Houray! Houray!

A sad side-note however is that regular macbooks with Intel videochips and mini-dvi adapters still don’t seem to work properly under XP. Maybe under Vista, who knows?

Okay, anyone tested MacBooks (non-Pro) on Leopard? Is there a way to make this work?

Certainly, booting the Mac side for Quartz Composer, Final Cut, Motion, VDMX, then switching to Windows for Resolume, vvvv, games, etc. has some serious appeal as a visualist dream setup. (Overkill? Sure. But a lot better than juggling machines or being forced into the OS by the hardware.)

Create Digital Music got an interesting comment about allegedly, what really happened to Java in Leopard. Short version: nearly all of the development team is gone; the one Java 6 person left and many others were moved to, of all things, iCal. Java 5 is likely to be all you get on Leopard, and still without improvements to JavaSound or QuickTime for Java, which have left sound and video support pretty weak. (QuickTime for Java is an even worse choice on Windows, though at least on Windows and Linux JavaSound runs better.)

Caution: this could be no more than a rumor. There seems to still be a Java development team at Apple, so at the very least, it sounds exaggerated. However, it’s also clear that Apple’s commitment to developing Java on the Mac is tenuous to say the least.

For Processing fans, though, there’s various good news. First, Apple has updated Java 5, the version of Java targeted by Processing currently. Secondly, while JavaSound still lags behind major quality improvements made on Mac and Linux, if Apple really is abandoning Java on the Mac, it could mean the finger pointing is over and finally Sun could develop Java on the Mac platform, bringing all three desktop OSes in sync (sorry, Solaris, that’s Linux, Mac, and Windows for nearly all of us). Another interesting tidbit: JavaFX, if Sun is to be believed, could be more than a scripting language, it could provide better multimedia support long lacking in Java. I imagine that could be incorporated into Processing. See the CDM story for more details — taking them with a grain of salt, as none of this has been officially confirmed.

Rumor: Mac Java’s Demise is Real, and Why That Could Be Good News for Multimedia

Click to play

Mrmr is an open protocol for mobile devices. It is used to dynamically create user interfaces on your iPod Touch or iPhone which respond to client apps in a multi-user performance environment.

Okay, that sounds awfully dry. Let’s try that again.

Mrmr lets you control Quartz Composer applications (or really , any compatible OSC implementation)over Wi-Fi from your iPod Touch or iPhone. Now you, too, can dance around like a lunatic while still controlling your visuals from the dance floor. Did we mention it’s multi-user, as well?

Mrmr is the brainchild of Eric Redlinger, researcher-in-residence at Brooklyn Polytechnic University’s Integrated Digital Media Program. He has leveraged the iPhone’s OS X underbelly and 10.5’s new Quartz Composer features to allow this sort of functionality.

I had the lucky* chance to interview Eric and ask him a few questions about Mrmr and the iPhone. Apologies for the quality of the interview, it was very spur of the moment.

Mrmr is a work in progress, but I think the results so far speak for themselves.

*(ok, no so lucky, my desk is right next to his, but somehow I had not seen Mrmr in action until just recently…)

We already knew that Leopard would be a major new OS release for Apple in the visual sense with Core Animation, much as Tiger was with Quartz Composer, Core Video, and Core Image. We should know a lot more at Apple’s developer conference in June. There are some additional details in Apple’s latest WWDC announcement, under “Leopard Innovations”:

1. Core Animation to add elegant animations to your application interface.
2. iChat Theater to incorporate application content in iChat.
3. QuickLook to provide system wide previews.

Core Animation is old news, but could this mean you could add iChat streaming to an app? And system-wide previews are also badly-needed.