Microsoft Research have added live, collaborative visuals to their bag of tricks for the Microsoft Surface multitouch table. Of course, in the process, they’re really demonstrating not only what you might to with Surface but with multitouch interfaces in general. In fact, it’s particularly odd that Microsoft hasn’t apparently made the connection with more generally-available multitouch hardware coming out, particularly with multitouch APIs built directly into Windows 7. HP is already shipping a mainstream laptop with a convertible, tablet-like form factor. And I don’t need to point out that this could lead to cross-platform, open source applications, not just those that run at Microsoft tech demos or on a unit installed in Vegas.

Via Ars Electronica:

VPlay: live video mixing meets Microsoft Surface (Subhead: “It’s like VJing on a Microsoft Surface!” Uh… actually, more than just like that, I’d say it is that.)

Thanks to Pedro Marques (VJ Danger) for the tip!

The ideas here, if in basic prototype form, are already interesting. The design is heavily influenced by (if not a direct copy of) the Reactable, down to the connectors between nodes.

http://mtg.upf.es/reactable/

Visuals arguably work even better, though, because they can be shown directly in a way sound cannot.

In this prototype, you can’t do much that you can’t already do with conventional visual software. But already, there are two significant, fundamental advantages. One is, having a nodal visual interface gives you really open-ended possibilities for setting up a set. Conventional software also relies on you to configure the modules you need in advance of performing. With this interface, you do it all live as you go – just as you can do with sound on the Reactable. Secondly, as the video points out, you can collaborate more easily, without fighting over knobs on your MIDI controller.

I couldn’t help but laugh at this particular frame from the video (which also, amusingly, heavily features Resolume v2):

Yeah? Speak for yourself. Us VJs are fighting the boys and girls off. We’re like fresh meat in a shark tank.

This solution is much better. Now people can get beer all over the inside of my multitouch controller. Give me the solitary existence any day.

As visualists, the sad truth is we have a poorer sense of the history of our medium than musicians. Part of this is simply a lack of access. YouTube is a weak substitute, but it’s a start. In that spirit, Karl (Format K) sends us the minimal geometric machinations of pioneering electronic graphics artist and animator John Whitney. We’ve previous mentioned the role of Whitney and Larry Cuba in helping the modern computer graphics industry to be born – with a little help from a movie called Star Wars. Here, you get a real sense of an artist working within the restrictions of the technology to produce something beautiful. It’s a chance to recognize how we’re indebted to this kind of work. While the temptation may be to replicate effects like this with more modern tools, they also illustrate how you can focus on a technique within a tool – and perhaps there’s a digital equivalent of focusing on artistic limitations.

The musical score turns this into a dream collaboration, with the work of Terry Riley.

It’s nice to have access to this, but boy, would I love to have an HD-quality rendition of many of these films available for download or on a high-quality medium like Blu-Ray. Any chance a modern-day Voyager would re-release seminal visualist work from decades past?

Computer vision for tracking movement is cool. But add the ability to track actual objects, and you can extend the possibilities for interfaces. We’ll be playing around with this on our upcoming tangible hackday. A lot of the reason these experiments can proliferate is the availability of free frameworks that make the technology accessible to artists and designers. The tricky tracking work is done, leaving you to focus on where this tracking might actually be useful.

The other good news: while doing projected visual feedback or fancier tracking can get more complex and costly, if you just want to track some objects, all you need is a USB or FireWire camera and some printed stickers. Cost: $40 with a webcam, about $5 without.

Recently, one of the most popular of these libraries got a big update: reacTIVision 1.4. It’s the open-source, multi-platform framework that powers the reacTable, and was developed by Martin Kaltenbrunner and Ross Bencina at the Music Technology Group at the Universitat Pompeu Fabra in Barcelona, Spain.

Multi-platform really means multi-platform. It works with FireWire and USB cameras, Mac, Windows, and Linux, and has clients for C++, Java, C#, Processing, Pure Data, Max/MSP/Jitter, Quartz Composer, and Flash, plus a wide range of applications that support the OpenSoundControl-based TUIO protocol.

You can grab the library at:
http://reactivision.sourceforge.net/

I’ll actually be testing both the tracker in reacTIVision and the Trackmate tracker from the LusidOSC project. The Trackmate software is built in OpenFrameworks. It does use a different protocol (LusidOSC), but that’s also based on OSC, and there’s even a tool that translates to TUIO.

For an example of what this all looks like when assembled - and some of the power of having a framework on which to build - here’s a tangible interface for a multiplayer game. It’s Pong with objects.

This games uses reactivision software, along with Flash, to detect symbol fiducial block movements. The game is played by moving these symbols on a table. Players can enter and exit the playing field at any time. The game adapts to the number of players. The lower the score the better; the first player with a score of 12 ends the game.

Music: Waterdrops by Yohan Shin http://www.geocities.com/cerup2

There’s more progress coming in reacTIVision world, too. First up: reacTIVision 1.5. Martin tells us:

After this release I am now implementing reacTIVision 1.5, which will improve the multi-touch tracking performance, and already implement the upcoming TUIO 1.1 blob tracking extensions, for the transmission of basic untagged object descriptors.

Following that, the next plan is for TUIO2, an expanded protocol that will address some of the shortcomings of the first version, to be released with a future update to reacTIVision. You can read the full specification for the new protocol, but Martin has kindly given us a Cliff Notes version:

To summarize, TUIO2 has a flat profile, which now includes symbol, cursor and blob descriptors in much more detail. Symbols now also can carry content info (e.g. datamatrix), cursors have additional properties such as type, pressure and region of influence, and blobs can be described in various incremental messages that describe the bounding, contour and skeleton for example.

The other important thing that Martin is doing - and the reason for the wait - is to synchronize implementation of TUIO in other key libraries and clients. That is helping keep TUIO a standard for this kind of work. It’s not even really a full protocol - part of the beauty of it is that it builds on OSC.

If this isn’t quite making sense yet, stay tuned and we’ll show some of the specific applications and get you started with your own projects.

If you’re interested in getting involved in our tangible interface hackday but had trouble filling out the form, here are the corrected links. Somehow I copied the wrong form keys yesterday.

RSVP in New York: New Work City Hackday / Party

RSVP anywhere you’re working in the world (no commitment, just so we can keep track of who’s interested, help you promote group events, and keep you informed)

International update: So far, Berlin, Canberra, Vienna, and India are interested in participating – and that was with the forms pointed in the wrong place! I think we can expect more. I’ll get in touch with you soon.

http://hackday.noisepages.com

There’s no supporting lesson to go with this clip, just an extremely good animated music video. Enjoy!

(Warning: Stylized cartoon boobies.)

WOOD from mc bess on Vimeo.

Thanks Karl.