Musicians have thousands of years of history when it comes to interfaces and instruments, but visuals are relatively new. Little wonder, then, that visualists are eager to try new interfaces to help make visuals akin to performance instruments. Or, in less lofty terms, let’s get Wii remote wagging in the club tonight.

Over on createdigitalmusic.com, we’re celebrating Game Day — basically, I’m squeezing as many game-related posts into 24 hours, because a whole bunch of tips came in at once. A couple of Wii-related controller solutions jumped out.

Wii + Flash

Getting hardware control in Adobe Flash requires some work: basically, you need a client-server model. The good news is, there are already two Wii-specific solutions out there.

MoteDaemon, Mac OS X
WiiFlash.org, Windows Blog, Download, Google Code

Pretty soon, people are cooking up Minority Report demos with Flash (using Papervision for 2.5D-style 3D in Flash’s 2D world, and Open Dynamics Engine for physics):

I’d loved to see this coupled with something like Onyx for an all-Wii, all-Flash performance app. With Flex and AIR (Adobe Integrated Runtime), this could be the basis for some really hard-core, full-blown apps … though you will be limited by Flash’s slower performance, at least in comparison to C/C++-based tools like Max or even Java.

Looks like Linux users are presently out of luck, unless I’m missing something (feel free to chime in if you know a cross-platform alternative).

Wii + MIDI, OpenSoundControl (OSC)

For a more app-agnostic solution, you can hook up a Wii to send MIDI or, ideally, OpenSoundControl (OSC) to apps that support it (vvvv, Max, Pd, and hopefully VJ apps soon — I’ve heard a couple of developers working on it).

On Mac, you can send both OSC and MIDI with one app, perfectly-suited to taking data from the Wii controller:
OSCulator. OSCulator is a great tool for the Wii, but it also shows promise of what a hardware input hub could generally look like, with open-ended inputs controlling visuals and sound rather than pre-defined, MIDI-style keyboards and knob and faderboxes as have traditionally been used in music.

On PC, GlovePIE is an awesomely-powerful scripting tool for use with Wii and other game devices. I’ve talked about it endlessly before, but I’ll stay quiet — just go. Get it. Enjoy. In fact, with OSCulator on Mac and GlovePIE on Windows, it’s hard not to be insanely happy on either platform.

Another interesting out-of-the-box alternative, though, is the new Wiinstrument. It’s largely geared for playing back drum samples, but it’s not hard to take that metaphor and use those control changes and other MIDI messages as visual controls — especially if you think of your “drum kit” as sets of visual clips (video or other visuals).

Wiinstrument on createdigitalmusic.com

If you’re using a Wii on Windows, VJ Kung Fu has posted a full walkthrough:
Wii to MIDI Windows Walkthrough

And here’s an example on Vimeo of using the Wii with Processing: (I’m sure there are others)
Wii + Processing

For more on the creation of the new VISP visual app, there’s a terrific story by creator Michael Creighton at Adobe Edge:

Building a visual performance app with Adobe AIR, Flex, and Flash

The advantages of this combination are potent for visual development:

• File system API (so you can easily manage video clips, generative sketches, whatever)
• Native window system API (so your UI looks at home on your OS)
• Flex, for CSS + XML goodness
• Clarity of the ActionScript 3.0 language (I’m totally onboard with this one, frankly, despite complaints from AS old-timers)
• Ease of development in Eclipse

As I noted, it looks like MIDI support comes from an external application. But there’s hope for more interesting possibilities in the future, by hooking up VISP to Java via a new AIR-to-Java bridge called Artemis:

Artemis Tutorial

Now we’re talking. It seems Michael is thinking like I was: wouldn’t it be great to have Processing hooked up to all of this? Now, of course, I hoped to do everything on the Java side, though there is that pesky problem with video support in Java being, how shall we say, less than fresh. But, in the meantime, I’m happy to combine the two where it makes sense.

Onyx was a little shorter on the AIR stuff (aka Apollo), which is what lets you manage files from the file system, so I could see some of what’s in VISP benefiting Onyx, as well .. and visa versa.

Now, someone in Javaland: you know you really want to improve Java’s multimedia support. Come on. All the Adobe people are having so much fun. They’re going to have much better parties. Think about it, won’t you? (Yes, unfortunately, Apple is one of the players who could help remedy the situation and they’re extraordinarily unlikely to be interested, but surely there’s a way.)

We’ve mentioned VISP earlier in the year, an open source VJ app based on Adobe’s Flex and AIR (formerly Apollo) Rich Internet Application platform. Despite some delays, VISP has been released. Check out VISP-VJ.com for downloads and source code. VISP’s home page contains tutorials, introduction videos and guidelines to download the latest frameworks required by the app. Geared towards generative real-time visuals, VISP looks be be a pretty exciting release, especially considering its open nature.

Some features straight from VISPs homepage:

• Works on both Windows and Mac platforms.
• Outputs a max resolution of 640×480 to a second screen.
• Supports MIDI input via a separate Java application (watch the screencast). Ed.: This is necessary because MIDI input remains impossible with Flash/Flex/AIR; you can get data in via XML sockets, but that requires Java — or something ese — to do the actual MIDI. -PK
• Emphasis on performing the content rather than managing a complex user interface.
• Use transitions to more seamlessly move from one piece of content to another.
• Use up to two filters simultaneously to add post-render effects to your output.
• Create your own custom content, transitions, and filters with ActionScript 3.
• Geared toward generative content, not video clip triggering.
• Source code is licensed under the GNU General Public License v2.
• Intentionally feature-deprived to impose constraints, allowing you to focus on your content and creativity.

We’ll be sure to give VISP a good one over and report back as soon as we can!

Ed.: Cool, but I’m still also interested in Onyx VJ. It’s also Flash/Flex-based, so much of what’s here is still true (though I think these are perfectly usable for video triggering). But Onyx has a lot of additional filters and features, including one very nice one: video playback from a VLC server. Of course, nothing stops you from trying both, since they’re both free.

Now, as for generative visuals, seems it’s time for the Processing community to respond. There’s nothing stopping you from doing something similar with Processing and Java, and since they can take advantage of OpenGL/Java3D 3D visuals, among other features, you’ve got more flexibility than with Flash. The main thing is, building a Java app to host other Processing sketches introduces a fair bit of complexity, so developing something would take some work. I’m doing some more limited experiments for my own performance with Processing; if any of it is useful, I’ll be sure to share. -PK