There’s a reason for Processing’s popularity. By making code simple, elegant, and direct, and catering directly to the kinds of visual ideas creative people have, the environment has made programming accessible to artists and designers in a way nothing else could.

Coding no longer has to be a source of fear, or a bad word.

But Processing, years into its life, has also badly needed a refresh. 2.0 is more than just a house cleaning. It’s a new direction, with “modes” that mean it’s no longer tied to old desktop Java architectures. (See a tantalizing screenshot below – with the click of a menu, you can make CoffeeScript JavaScript for the Web, or run on Android.)

I still love Processing as a way of sketching out ideas, and with strategic use of the GPU in its now-native OpenGL rendering, it can also be surprisingly high-performance.

Of course, that 2.0 reboot has been a long time coming, enough so that you might have even forgotten it was enroute. That’s why the recent 2.0 beta 9 is big news. It includes some major new features that finally reveal what 2.0 is all about – and bug fixes that have been a long time coming. In fact, it’s that moment when the betas stop looking so much like betas. Here’s what to expect.

New in this version: an all-new interface. Things do look prettier, including a new icon, but it’s under-the-hood changes that matter more. You can install a command-line tool for building Processing sketches easily with any tool you like – such as your favorite text editor. Modes, now as separate projects, hold the promise of getting speedier updates than they would if everything had to be rolled into Processing proper. And a succession of changes make things like finding examples easier — some kindly rolled retroactively into the stable 1.5x sequence, but which, added together, make this a very friendly environment.

Other major changes accompany this release:

• An official JSON library.
• MovieMaker finally works again, for recording your work. (Syphon and that library will work, too, of course, but this is nice to have anyway.)
• Vastly improved and streamlined data classes – good news for data visualization and the like.
• 2.0b8: Windows 8 and Retina Display fixes.
• 2.0: Built-in video library that really works (and, again, with the GPU can actually work really well – think HD mixing on lower-end machines).
• 2.0: Core OpenGL library for all rendering – with super-fast results.
• 2.0: Table support for easy data processing, plus improved XML support.
• 2.0: Native 32-bit and 64-bit support.

For more detail, see:

An overview of what’s changed in 2.0

A detailed changelog with every little bug fix and improvement

I think it’s the better integration with the community that may make the biggest different, which we can see visually.

For instance, the update manager now automatically installs supported libraries as they’re improved:

The mode manager supports targeting things other than just desktop Java – addressing one of the biggest complaints about Processing as the years have ticked by (though, I will say, it is useful having Java library support, and that can still deliver searingly-fast performance when you’re doing mostly OpenGL things):

Oh, and there’s a new letter “p.”

processing.org/download/

4 ways to sync VDMX and Ableton Live from Studio Rewind on Vimeo.

Frequently asked question? Maybe incessantly-asked question.

Live performers simply want a way to have more intimate relationships between music and visuals onstage. That means whether they’re working solo or with a live visualist, being able to get useful signal between music and visual tools and performance elements.

Livegrabber has got to be about the easiest way to do this I’ve seen yet. It’s actually a suite of plug-ins for Ableton Live’s Max for Live environment that spits out OSC (OpenSoundControl) messages to any visual tool that can respond.

And as you can see in the video, the results are both effortless and profound.

It’s best seen in this example with VDMX. (Any OSC tool will work – I’m rather keen to code around this with Processing, for instance, just for kicks. But speaking of VDMX, that superb tool is on a fire sale for a hundred bucks this month; act fast!)

The basic elements:

Sender: sends a bunch of OSC
Receiver: receives a bunch of OSC
ParamGrabber: here’s where things get interesting — connect to any parameter in Live
TrackGrabber: here, pull track-level parameters (like triggering clips)
AnalysisGrabber: this does more conventional audio spectral analysis
VoidGrabber: perhaps the most fun, you can use Automation and Clip Envelopes to send discrete information output for visuals
SingleNote: here, you can dig into individual notes – but with assignment that could theoretically be easier than simply using whole MIDI streams (which can be too much data)

The ability to go fine-grained, for solo performances or instances that need lots of synchronization, all the way to larger grain, when you just need particular cues to pass off to a visualist, seems really nice. And with Max for Live now in Live Suite and on more Live machines, this should also be a boon to audiovisual collaborations.

And if you can find a live visual performer with whom to collaborate, it gets all the more powerful.

Now in free “beta” (no word yet on what they’re planning to do with pricing long-term):
http://showsync.info/index.php/tools/livegrabber/

Lots of other amazing-looking Live-based visual/lighting tools, too many to mention:
http://showsync.info/index.php/tools/

Thanks, David Lublin and Dave Gutt for the tip! (Dave endorsed!)

Call it laser mapping.

Melbourne-based artist Jayson Haebich has rendered in thin air, literally, architectural forms in color. He uses custom software to map lasers through particles to produce an ephemeral sculpture of air. The results are gorgeous – frozen digital motion.

From his description:

These are a series of static light sculptures that have been created using laser light, smoke, shadows, physical shapes and custom built software to create complex compositions of shadow and light that play with the sense of depth and perception. These pieces challenge the observers idea of perspective and ask them to consider what components of the installation are physical objects and what are non physical objects such as light and shadow. These sculptures fill the room with light and colour creating an almost mesmerizing effect as the beams of precisely mapped light from a laser cut through swarming particles and haze suspended within mid air to create almost solid looking planes of illumination.

These pieces were created using custom built software that is used to map out physical features using a RGB laser and was created using OpenFrameworks.

He tells CDM more about his intentions and inspiration:

The main motivation behind these pieces is to use non-physical elements such as light and shadow as components of a solid-looking sculpture. They are quite simple in design: I place blocks and columns throughout the room and then map these out with my laser software. This enables me to create several effects such as having the columns stop the laser beam directly, or having the columns cut the laser beam in certain spots which, due to the contrast of the very bright laser, creates dark shadows that themselves look like they are made of some kind of physical object. There is also a smoke machine, which makes the laser beam easier to see as well as enhancing the atmosphere of the space.

These pieces (and my other works) are heavily inspired by Anthony McCall‘s light works such as “Line describing a cone” and “Cone of variable volume”.

To me, this strikes at some of the essence of the reason we work in digital media. In these immaterial forms, he bends perceptions to their outer boundaries. Beautiful work; see the whole set:

http://www.jaysonh.com/vector.html

Take your visual app. Take a parameter. Now control it with timelines and automation – anything, anywhere.

The appeal is clear for visualists, particularly with built-in options sometimes limited. Now, sequencing control – clocked to MIDI – can take on powerful new dimensions.

Vezér isn’t the first app to go this direction, by any means. I’ve admired in past the insanely-powerful IanniX, for instance. But then, “insanely powerful” isn’t always what you want – particularly if that depth just drives you insane trying to do something simple. Vezér, just now entering beta, promises to be a bit simpler. And coming from the developer of CoGe, it also arrives from a coder with a track record.

We’ll be watching for this to come out, but in the meantime, its creator gives CDM a first look. Details:

Vezér is under development, no public beta available yet.
You can have any number of compositions and each composition can contain any number of tracks.
There are different track types in Vezér, like single Midi CC message or Midi CC range.
Vezér supports Undo/Redo in the whole application and also supports Copy/Paste of keyframes.
The playback speed of a composition is adjustable and can be synchronized with a Midi Clock.
The resolution – FPS – of a composition can be set.
Vezér supports sending of Midi CC messages.
Different interpolation can be set for each keyframes.
Vezér support recording of incoming Midi signals.
Vezér can be controlled via Midi or OSC.
Vezér is a 64 bit application for Mac OSX 10.6 or later.
Vezér will be a commercial application.

Developer Tamas Nagy explains that ease was part of his goal:

And maybe that is the key of Vezér. I really trying to make it very easy to use but also powerful. My goal is keep it simple at possible, I want to users feel its just a tool, not something other complex 3rd party software which needs a lot of time to setup That why I chosen MIDI support instead of OSC for the release, because it -almost – does not need to anything to make it working – but yeah, OSC will be supported in a future release for sure. And other important thing from my point of view, its not “just” a timeline, where you can set up when your clips will be triggered, but more likely a tool for setting up predefined MIDI actions for your performances. Likely in this article, but without recording – sending actions to applications:

RECORD AND PLAYBACK MIDI IN COGE, MODUL8, RESOLUME AND VDMX [Vezér Blog]

Ah, color me interested on that. Keep up with the latest on the new app site:

http://www.vezerapp.hu/

Body mapping and dance/visual fusions are still explored only in fits and starts, compared to the extent of live music and visual performance in other media. So, it’s encouraging to see this latest experiment from dancer Christian Mio Loclair. Working with Microsoft’s Kinect, the slowly-undulating tendrils of visuals behind him create visual counterpoint for headstands and hip-hop dance techniques. Far from running up against latency, here there’s a sense of visuals that answer the moves with a slow sigh, creating a kind of living architectural space behind him.

The tools:

• ofxOpenNI and ofxCV work with OpenFrameworks to analyze imagery from the Kinect camera.
• MSAFluid for OpenFrameworks by our friend Memo Atken again provides visuals (actually, some of you probably already spotted it)
• ofxUI adds the UI.
• vvvv (“V4″) is used for calibration; see Christian’s blog post for more tips on making the most of calibration technique, and don’t miss the Elliot Woods tutorial that inspired him.

Christian muses to CDM, “I am very convinced that especially the Kinect and the upcoming Kinect 2 will change the way dance will be performed. I hope I can contribute to this development … just some streetdancers, hackers and a Kinect. I wanna see how far we can get by open source Code, own code and Open Dance.”

More:
flow no. 1 | kinect projector dance

Blast from the past: this color organ is from 2007. But it’s a beautiful demonstration of light and sound, fused into a single interface, and thus worth mentioning as I pull together notes for a talk at Mapping Festival tomorrow here in Genève. Compare the 60s-vintage Lumigraph of Oskar Fischinger, which I write about today on Create Digital Music.

In gooey pinks and purples, traced with imaginary sparks, the game controller-manipulated system resembles looking into the heart of a great jellyfish made of plasma.

Lightbender v2 – audiovisual instrument from Sixth Sensor on Vimeo.

From the notes, as well, are some nice thoughts and references on other “painterly” av interfaces and color organs, including the PhD dissertation work of none other than Golan Levin.

Worth reading:

Lightbender v2 is an audiovisual instrument developed for four players each controlling the expression of a color and a tone. It was exhibited at the contemporary music festival Sound Around (2007) in the Plex music theatre in Copenhagen.
The instrument incorporates a half sphere shaped water tank that serves as a semi-3D video screen. A particle system is projected into the tank leaving illuminated traces in the water. The sound is based on an additive fm synth with filtered white noise. Synth parameters are mapped directly to the behaviour of the particle system. The sound is emitted from four speakers in a surround set-up where the panning of the sound fits the position of each player and the movement of the particles. The physical interface consists of four modified game pad controllers.
The first version of the instrument which was built for the exhibition Social Aktion (2007) at The Museum of Contempoary Art in Roskilde had only visual output. The initial idea was to engage people socially in a musical experience by enabling the control of visual expressions while listening to a selection of the museums extensive sound art archive.
Building color organs is no novel idea. The first in this family of instruments is the Clavecin Oculaire completed by Louis-Bertrand Castel in 1734, but the idea of audiovisual coherence dates even further back (Peacock 1988). Since then many have followed, most notably (I think) Thomas Wilfred’s Clavilux from 1919.
I’d like to thank Golan Levin for compiling a brief a historic overview in his PhD thesis from 2000. Thanks also goes to Morten Søndergaard, for the invitation. Museet for Samtidskunst, for providing materials. Daniel Høier Øhrgaard, Morten Carlsen and Thomas Sørensen for general help. Marie Louise Andersson, for game-pad modification help. Michael Edinger, for help with the electronics. Enrico, for lots of practical help at the museeum. Dennis Paul, for teaching me a bunch of Java tricks. Christian Riekoff, for providing the procontroll library.
Peacock, Kenneth: Instruments to Perform Color-Music: Two Centuries of Technological Experimentation. Leonardo 1988. Vol. 21, num. 4, p. 397–406.
Levin, Golan: Painterly Interfaces for Audiovisual Performance. PhD thesis MIT 2000.

The project is the work of Copenhagen-based Sixth Sensor, aka Carl Emil. Loads of other projects to explore of his, as well.

Lightbender

Nail the finger fireworks of a particularly hard Rachmaninoff, and you may well feel like blasts of light are shooting out of the piano. But to give the audience the same sense, a DIY instrument made of cardboard and homebrewed responsive lighting translates that keyboard virtuosity to an optical show. Reader Aylwin Lo sends us this project out of Canada:

Rachmaninoff’s Étude-Tableaux Op. 39 No. 6, as performed by YT//ST’s Brendan Swanson from Aylwin Lo on Vimeo.

I’m with YAMANTAKA // SONIC TITAN. We’re an art collective based in Toronto and Montreal that is most known for making music and putting on dramatic live shows. People like Pitchfork, Vice, and MTV Iggy have nice things to say about us.

We made a video of our keyboard player pulling off a notoriously-difficult Rachmaninoff composition on a special piano we constructed
from an electric piano, a cardboard baby-grand shell, and a homebrew, Arduino-based LED-controlled light rig, and we thought you might like it.

You thought right. Now, if you want to play piano like this, you … uh, have some practicing ahead. But in a novel twist on crowd-funding rewards, they’re also using this very artist to help – even as they work on a video game. (Stay with me here.)

We’re partly doing this for an EPK we’re putting together, but also to promote a perk on the Indiegogo campaign we’re running to finance the development of a videogame we’re working on, called YOUR TASK // SHOOT THINGS. That perk is piano lessons with our keyboardist:
http://igg.me/at/ytst

Clever.

Here’s a look at the game.

YOUR TASK // SHOOT THINGS Trailer from Aylwin Lo on Vimeo.

Music and gaming, sound and immersive lighting converge again!

Entropy never looked this good. Or, certainly, it looks a lot better than when I broke that beer glass. (I know, I know: there are reasons why we don’t want to live in a universe where doing that would make the glass re-assemble itself from its fragmented shards.) Photo courtesy the designers.

In an elegant, balletic dive, taking an almost impossibly-long span of time, a single droplet of water falls and splashes, an animated logo peeking out from the inside. But it’s what isn’t there that may surprise you. There’s slow motion camera behind the scenes, meaning the usual way of doing this is absent.

Instead, what you’re seeing is a stop motion time lapse – a record of the shifting patterns of entropy in nature, thousands of different droplets appearing as connected that in reality are not. It’s a trick of animation and high-speed lighting, not high-speed photography, stroboscopic illusion.

And it’s a fun DIY project, to boot. 3D and robotics? Okay, we’re in.

The Barcelona-based studio that produced it shares not only the animated result (including a logotype for design mag IdN, but also a short film explaining the making of. Physalia’s Belén Palos writes us:

We got your contact through Alex Trochut, who shares studio here in Barcelona with us. We are big admirers of Create Digital Motion, so we would love to share with you our new piece Entropy, a joint effort from our 3D division and robotics Lab, in which we created a system to capture the fall of a water drop without a slow-mo camera- with replacement animation mapped inside the drop! There’s an Arduino involved, as well as our self-developed Motion Control.

Watch the results:

Entropy – IdN Magazine v20n2 from Physalia Studio on Vimeo.

And then the making of:

Making of Entropy – IdN Magazine v20n2. from Physalia Studio on Vimeo.

For all the gory details, they’ve got a terrific post explaining how they came to this project and how they pulled it off:

Entropy – Long version [Physalia Blog]

In brief, though, the ingredients:
1. A laser beam-photodiode, to detect the water.
2. A “Photoduino” Arduino circuit. (Be sure to read the blog post to avoid burning out optocouplers.)
3. A DSLR, the camera the trigger circuit will trigger.
4. A custom-built water course made by hacking a medical drip.
5. A motion control rig that tilts the camera’s lens with the water.
6. (My favorite part) a servo whose sole job is to tap keys on the keyboard for serial communication via Arduino. And it’s inspired by Homer from The Simpsons – the best kind of hack.
7. A whole mess of Arduino code.
8. Some conventional digital animation, then mapped to the photos.

Fantastic. Hope we see more from this crew.

Mapping: it’s kind of everything. It’s the projected image mapped to the surface. It’s pixels mapped to lights. It’s the control layout you use on your iPad and your fader box mapped to parameters in visual output. It’s the translation of music to lights. It’s the range of color on the filter. You’re constantly mapping one thing to another.

And of course, the community of people who read this site are generally somehow undertaking the difficult task of mapping across media, as you map visual performance to music.

So, it’s fitting that “mapping” at Mapping Festival is about more than projection mapping, as the Geneva, Switzerland audiovisual happening kicks off this week.

“Visual audio” and “deviant electronics” are the theme. That ranges from dynamic club music from the likes of Answer Code Request, Jon Hopkins, and Clark, to Onionlab’s architectural audiovisual experimentation and CENC’s fusion of dance with visuals.

I want to talk a bit about musical mapping. For a panel I’m leading at Mapping on Saturday May 11, I’ve pulled together a few of the artists at the festival who deal with that issue of musical translation and image. Each of these artists works at that touch point of sound and picture.

Sougwen Chung has done so with her collaborations with Ghostly International, but in Geneva turns her work to an installation that responds to sound from the space, transforming a room into a responsive, fluid architecture fed by ambient noise. See image at top.

Wraetlic, by Alex Smoke and Arch Project.

Alex Smoke‘s Wraetlic works with live performance, Jitter-powered visuals by Japan’s Arch Project made to respond to and accompany his music. I saw Alex playing the work live at Berlin’s Berghain, which means – ah, I have no images. Instead, we can see what it looked like in Montreal. Electric geometries flailing to the beat in the background is nothing new, but having that added dimension for me added to the ability to go into trance to Alex’s deeply emotional live music. (And yes, I could largely dance and watch, happily, aided by Berghain’s massively-powerful sound system. What that setup lacked was good projection to match the sound, so I can’t wait for Switzerland.)

See some informal performance documentation, a music video, and an interview with Alex Smoke.

DIGITALIS: Alex Smoke presents Wraetlic from MUTEK on Vimeo.

ATOM™ (Uwe Schmidt).

ATOM™, aka Uwe Schmidt, perhaps needs little introduction. He is an icon of the electronic music scene for going into a fourth decade now, and represents the masterfully-focused taste machine that is Raster-Noton. Aesthetic sense and sensibility permeates everything they do visually and sonically, from the sound of the tracks to the look of the packaging. HD is his latest audiovisual solo work, in for its European premiere. And… well, that’s all I can say about it, as I haven’t seen it yet, but I’ll have the chance before we have our public conversation. And it’s worth hearing Uwe’s thoughts on visual and musical connections any day.

These teasers give you a sense of the sound, if a limited sense of the visuals.

Full description from the panel below I think should give some idea of how I think about these things. And I’ll talk a bit about my own work and the way we might approach bigger issues.

I hope we’ll have more to bring you after Mapping. If you’re in Geneva, say hi. And if you have thoughts on this topic – artists or otherwise – please do be in contact.

Musical Mapping: Lecture and Discussion

I. LECTURE
At the Meeting Point of Image, Space, Interface, and Sound

Mapping music to visual form has been a metaphysical challenge that has enchanted designers, architects, and composers alike. Audiovisual artist and journalist Peter Kirn traces some moments in the translation of music into image, in architectures of “frozen music” and notated scores reimagined as interactive visual media. He then looks at how tools can merge visual and musical media into a single, integrated interface, including a series of examples from his own work in the free tools Processing and Pd. These suggest the possibilities of expanding audiovisual performance, and “unfrozen,” fluid scores and architectural transformations of music.

II. ROUND TABLE
Rhythms of the Visible, Image of the Unseen: Audiovisual Technique and Musical Design

Following this introduction into some of the ideas behind audiovisual art, Kirn will moderate a lively discussion of artists from a number of different backgrounds who are presenting their work at Mapping.

Alex Smoke will talk about his live AV show at the festival, and integrating the performance of images and music. Sougwen Chung works across media ranging from interactive installation to traditional, non-digital illustration. She has collaborated with Sepalcure (Travis Stewart/Praveen Sharma) and the label Ghostly International to design visual interpretations of music, both hand-drawn and in live digital performance. Producer ATOM TM brings a musical perspective from a label that has consistently interwoven design and music, and as an artist works across a range of visual interfaces for music, and will talk about his HD project getting its European premiere at the festival.

We will talk about how design, illustration, and performance can realize musical ideas in imagery, and what visual metaphors in art and interface mean for our conception of music.

Peter Kirn (US/DE)
Alex Smoke (UK)
Sougwen Chung (CA/US)
Atom TM (Raster Noton, DE)

http://www.mappingfestival.com/2013/en/program/conference/bac/musical-mapping

Sorry, weird to quote myself, but that’s the way I’m describing the event. Stay tuned – next week is one I definitely anticipate, getting to hang out with some of the people who inspire me.

http://www.mappingfestival.com/

Motion, light & sound / Kinect & Madlight from things happen on Vimeo.

In a teaser video just released by Spain’s Things Happen, a silhouetted performer uses arm position to sweep through RGB colors and trigger sound cues. It’s the latest effort to integrate the immersive media environment with a performer’s body, part spectacle, part interface.

The ingredients, apart from Microsoft’s ubiquitous Kinect depth camera:

Motion capture + image = light + sound
Software:
MadMapper [using MadMapper's Madlight feature to trigger lighting]
Quartz composer
Ableton live
Music: Sun Glitters x Isan – Snowfall

The nice thing about the inter-linked, comment-enabled Web is, we get to see more.

Inverting the relationship of color to output, Things Happen uses a color turntable to trigger lights and sound. The colors themselves become a score for audiovisuals, a return to the days of optical discs and optical film tracks.

Light turntable test. from things happen on Vimeo.

Deets:

Real time experiment with video, audio and RGB led light.
Software:
Madmapper (Madlight)
Quartz Composer
Live
Hardware:
2 RGB led strip
Led controller
Enttec ODE

This sort of light immersion we’ve seen in commercial products and experiments alike. So, through Vimeo comments, we’re also treated to this “Lightosphere” project, combining interaction and RGB light. It’s the work of Tobias Schmid und Michael Nef of Bern, Switzerland’s Captns & Partner. (Mmm… 1×1 pixel color! Via the Italian site Interactive Design.)

Interaction Design > Lightosphere from Captns & Partner on Vimeo.

Lots more on the Facebook page for that project.

It’s lovely in a way to see the convergence of these projects. Far from disheartening designers, I hope that this sort of resonance should encourage pushing boundaries (in the spirit of competition). Also, speaking as press here, critical mass is often what an idea needs to take off – particularly with installation-specific projects that by definition must take place in only one location, as a global audience grows in their appetite from this work.

It also suggests that, perhaps, you’re not crazy – that someone’s on to something.

Keep iterating. And keep magically making pretty lights happen with your hands.

Seen other projects, or worked on one of your own? Let us know.

Thanks to Tom Phillipson for sending this in.