SND @ Club Transmediale 09 from Wuestenarchitekten on Vimeo.

You’ve had those moments in which the dock of your Mac popped up or the mouse became visible. None of us enjoys that. But what about when the interface itself – the UI that’s directly related to producing what you’re trying to express — becomes visible? To reflect on that idea, we head out to a very cool event at Transmediale, so we can — you know — transmediate.

Sometimes, layers of goodness converge into an awesomeness event horizon. And there are some delicious flavors in the event itself. So, where to begin?

  • TouchDesigner is a striking, modular GPU-based video + compositing + 3D app for Windows that does … well, everything. It’s “node-based” or patching-based. There’s a free-for-non-commercial use version with slightly fewer features. (Thanks to our mate TweakingKnobs for calling this app to my attention.) But aside from the fact that the tool itself combines everything you might want …
  • Markus Heckmann / Wüstenarchitekten, showing off TouchDesigner in this case, is a wonderful live visual artist. The technology here is really adeptly used by the artist/developer. And the choice of tool is, to me, essential to the event that results.
  • SND and Alva Noto make perfect sonic collaborators, with stripped-down, raw, elemental digital sounds.
  • Together, they join to form the CTM.09 performance at Transmediale 09, which the presenters describe as an audiovisual “infinite dancehall.”

Our friend Greg J. Smith (Serial Consign) writes about all of these folks more eloquently than I can. Aside from how the collaborations interconnect, Greg notes that a moment in which you see the interface actually produces an entire additional layer of meaning to the performance.

About a third of the way into the video there is a moment when the “full bleed” visuals pull back and reveal that the graphic we were looking at is actually just one of many, nested in a larger interface.

I love these wide angle views where visual performance acknowledges its underlying systems of organization – this could almost be described as visualist marginalia. In moments like this, integrated AV performance is transformative and turns clubspace into a situation room, a self-monitoring synesthetic feedback loop.

Full-Spectrum Dominance | Serial Consign

Greg was, in turn, inspired by vade’s post last year on this site for us on KDaG_nato+0.55. The mysterious woman who invented that creation didn’t bother separating glitches, mistakes, UI elements, output, input, interface … it’s all one big jumble of visual information, but united by a relentless aesthetic.

This is quite nice, as we’re getting sort of recursive blog post loops going, and the beauty of TouchDesigner is that its interface is recursive. The UI is built in the same environment that makes the output, the output is the UI, you use the UI on the output, you can zoom into the UI to an output that … you get the idea. Instead of manipulating one screen to impact the result on another screen, you can layer your work inside a larger patch, and make everything visible as you go.

Greg wonders if there are other software tools that work in this way. But I suppose I’m as interested in finding ways of building UIs in your own creations in the same way. Instead of hiding the interface – or, alternatively, instead of trying too hard to reveal what you’re doing – there seems to be lots of potential to fuse the two and eliminate anything that isn’t strictly needed. Food for thought.

After the jump, more video examples.

Alva Noto @ CTM 09 from isabelle rousset on Vimeo.

Quite minimal stuff, but here’s more of TouchDesigner in action. Apparently, TouchDesigner has evolved out of a motion graphics package — Houdini — but has been rewired for real-time. And that’s exactly the sort of thing I hope we see more.


Particle Simulation in TouchDesigner to Pan Sonics Askel from their Album “A” from Wuestenarchitekten on Vimeo.

For more on TouchDesigner, our friend Todd Thille has a terrific write-up. Here’s a shot he took of the interface. So imagine being able to zoom in on a single node, and have that then reveal the next section of your performance.

Getting back in(to) TouchDesigner | todd thille

The software has been around since 2000, but then, I’m pretending like a lot of the last eight years never happened (for a range of reasons, not only political, but because it also means I’m completing my PhD REALLY FAST), so — hey, it’s NEW!

  • http://www.sparkav.co.uk toby*spark

    cool – i've been wanting to write up that touch demo we saw in berlin for cdm, but just haven't had the time. good to see it happened anyway. it certainly is more than impressive.
    http://tobyz.net/tobyzstuff/diary/2009/01/visual-

  • http://www.mutantquartz.com Steve Elbows

    Touch Designer's UI and price structure seems to have changed radically in the last year or so. Ive only dabbled briefly with the new version. I got a bit more in depth with an old version but was ultimately put of by price and uncertain future of the product, but that was years ago and its nice to see it evolve, it always had potential.

    So the price barrier to entry is now gone. The UI is a lot better, but there are still a heck of a lot of confusing terms & UI elements, and unclear documentation, that will put off most people. Which is a shame because for those who are brave and have plenty of time to invest, Im sure very interesting realtime stuff, both 2D and 3D, can be achieved with Touch Designer. If a couple of people could create tutorials to ease the pain, I think it has a good chance of getting a new following, I will certainly return to Windows to try it again if I ever have the time.

    As for other apps that have the same UI concepts that you highlight, I dont think Ive seen anything that does it quite like the newer Touch Designer. Some modular apps give a preview of the output at each node, or have nodes within nodes, but not quite how touch does it.

    I am presently messing with stuff that is partially related to this topic. I was looking at multitouch projected surfaces. The typical app for these has a very rich UI, and often the UI and the visual output are the same thing (eg particles following your finger).So I started messing around with quartz composer, to build a simple sequencer which is multitouch, and outputs 3 things: midi to generate music, visuals triggered in time to the sequence, and a visual representation/UI for the sequencer itself. In a live scenario I envisage multiple screens, eg the central one showing triggered visual output, and other screens showing the sequence visually, and also visuals of any realtime controller signals coming from live performers. Technically this isnt what you are talking about, because the actual quartz composer editor UI used to build the beast isnt shown to the audience, but most of the performance data that matters is. Im only about 5 days into this project so havent gotten too far yet, I just have a 1 track 16 step sequencer that has a UI made of 3D blocks and cylinders that responds to mouse and whacks out simple midi notes.

  • Chuck

    Thanks for this update on TouchDesigner – I too had last looked at Touch a few years back when the cost of entry was significant. So am definitely going to put some time into evaluating this. Can anyone speak as to how it compares to VVVV? That's the tool I've been concentrating on most recently – just in a tire-kicking mode.

  • http://www.mutantquartz.com Steve Elbows

    Im not sure how it compares for 2D stuff. For those interested in realtime 3D I strongly recommend giving Touch Designer some time, from NURBS to Inverse Kinematics, it offers a lot thats hard to find elsewhere.

    Actually what I couldnt find when I tried the new version, were many 3D demos, which is a tragedy. If this has been rectified I would love to hear about it, as I just couldnt get my head round it all without such examples.

  • massta

    I can't say enough about Touch Designer. I've been using it for a while now, but stopped during development of 077 to produce a Houdini Apprentice HD ($99) animation: http://www.vimeo.com/2405348
    I'm bringing this up because these two apps are very similar and can greatly enhance any artist's (non-commercial) production.

    Derivative Inc., like Houdini, has unbelievable support. Not only is everything documented in the forum and wiki, they're constantly updated. Want to see a feature or have an issue, just ask. New builds arrive all the time.

    I here you Steve about tutorials. Some are here already:
    First things first: http://tinyurl.com/d2bdlw
    and another (older build) showcasing node building visuals: http://tinyurl.com/d4fkjl
    with more to come, I'm sure.

    The real magic is with environment file sharing. To look at the schematic of another user's Designer file is the greatest asset. User notes are easily placed into any Node and can be found and read with one click. I do this for myself often to track what I'm building. Copy, Paste, Drag/Drop, work side by side with two environments open, or just load everything you've ever built into one environment, which is something you couldn't do with the older version and good reason it's been a while since everyone has looked at Touch. Well, the wait is over. Software users are like friends; you can never have enough. So check it out.

  • http://www.tweakingknobs.com TweakingKnobs

    wooooooooooooow thanx for the link to my web peter ;D

    Keep it up !

  • http://www.tweakingknobs.com TweakingKnobs

    The only thing i dont like about touch designer is , that it only runs in windows, wich software that desevs respect , in 2009 runs only in windows?

    And i dont want to desrespect touchdesigner but come on.

    windows ???

  • http://www.derivative.ca Isabelle Rousset

    We have uploaded more video documentation of the raster_noton / Derivative spectra at CTM09 which can be seen here: http://vimeo.com/groups/touchdesigner and here: http://vimeo.com/3390441

    Videos include performances by Byetone (Olaf Bender) performing 'Plastic Star', Frank Bretschneider, and an amazing piece by Olaf Bender + Carsten Nicolai ending off the evening at CTM09. An overdose of awesome!

    Check back for further updates.

    tschüß

  • http://www.imagineer.net.au/australian_vj asterix

    I rarely make negative comments but I really didn't get this. For an "audiovisual “infinite dancehall.” there wasn't any dancing and the visuals certainly didn't give a sense of infinity (more monotony). And the audio to visual didn't seem to have any apparent sync or much relevance to each other. Perhaps the crowd were cheering at the end when it finally finished.

    Certainly touch designer is capable of much more than this. Again I hate being critical but I hope this will motivate them to push themselves a little harder next time.

  • http://www.imagineer.net.au/australian_vj asterix

    p.s. – I missed the other two vids by the way and was referring more-so to the first. Love the particle system in vid 3.

  • marx

    Though the documentation is lacking, as stated above, Derivative has amazing support. They are always on top of everything you ask, no matter how dumb. Now that it's free I hope that more people check it out.

  • Pingback: Create Digital Motion » Gorgeous Timeline-Less Audiovisual Multi-Touch Sequencer, Built in Flash

  • http://experimentaclub.com/druhb Jaime Munarriz

    When playing with a Processing+PureData set, I felt the need to show all the interface operations going on, so I decided to use a DualHead, 1 screen for 'clean' visuals, the other for the dirty programming and interface tweaking. But a true interface with the visuals inside, that is really amazing!
    The only problem is that it's going to be difficult to do something with this tool that doesn't look 'Raster Noton's".