Toy theater moves from the epic facade of a grand performance space to something scaled, in human terms, to miniature. Projection mapping can do the same thing. Just ask artist Tonner Vi. As writer Erica Gonsales has chronicled for The Creators Project, the Colombian artist has though small – egg crate small. The results can be brilliant, with cleverly-conceived content. Here, that moves to a whimsical take on the home game console. (Warning: may cause a surge of nostalgia even for non game nerds.)

Lots of tools here, including CDM regulars Cinema 4D and Ableton, but Modul8 does the ultimate heavy lifting on the projection.

Description:

VISUATARI – 3d Videomapping installation
By Tonner Vi & veoseven
…
Visuatari is a work using the technique of 3D videomapping, in order to contrast this latest technology with classic sound and picture to video games, giving honor to that days, where children admired such technology could not fail to see and play in front of that virtual world so unknown and so interesting as well.
Enjoy !!!
…
Softwares-3dsMax.Cinema4d.Reason.AbletonLive.AfterEffects.Modul8.Millumin

More on Vimeo

This just in – a nice spot for the new iluzion visual label, which also features Tonner VI’s audio work:

Abstraction and geometry have their expressive qualities, but architectural projection as a medium need not always go that route. VJ Sauve’s work “Homeless” is poignant, warm, and personal, crafted from hand-drawn and hand-painted media, transformed into digital animation, then projection-mapped in a way that seems to bring São Paulo’s alleyways and streets alive. Characters dance and run across the urban landscape, with a convincing blend of simulated movement and real, moving projection. It really is cinema, fusing a story with the projection-mapped setting.

The making-of video, below, is an artwork in itself; bringing in the story of how the piece was made with the story of the film. Seeing hand drawings and paint to go computer, then venture out into the Brazilian night as São Paulo’s residents look on, is its own kind of theater. And you can add this tune to the emergency playlist you queue up whenever your mood has gone glum.

More information:

vjsuave.com

ygormarotta.com
podoboo.com.ar
facebook.com/vjsuave
Director: vjsuave
Art Direction: vjsuave
Character Design: Dante Zaballa, vjsuave
Character Animation: Dante Zaballa
Animation: vjsuave
Music: Juan Tortarolo
Edition: Guillermo Coube, vjsuave
Camera: Rafael Garcia
Camera assistant: Joao Maia
Mapping: vjsuave
Locations: Rafael Garcia, vjsuave
Producer: Juliana Borges
Camera car: Neto Valesi
Supported by MTV

And for the behind-the-scenes video at bottom:

Camera: Pedro Valente, Rafael Garcia, Joao Maia
Music: “Alegria” by Los Animales Superforros
Edit: vjsuave
São Paulo, 2011

Spotted tools: After Effects, MadMapper.

Side note: can we make this the new MTV? (Or move to Brazil, then do so?)

Based in Brazil, “Vjsuave is Ygor Marotta and Ceci Soloaga a duo that creates live visual performances, urban interventions and videos. We combine plastic art with digital animation, street art with “video mapping”, endless quests for love with urban interventions.”

In moments that would have pleased Belgian surrealist René Magritte, an ordinary paint roller transforms a litany of European landmarks with video “paint.” Roll on, and it’s as though someone has punched a hole to another dimension. This isn’t a post-production effect: it’s all live, via computer vision and projection mapping.

The project is the latest work by international duo SWEATSHOPPE. We’ve seen them twice before:
Painting with Video: Live Projected Wheatpaste with SWEATSHOPPE
Hypnotic Colored Geometries and Balloon AV from SWEATSHOPPE

SWEATSHOPPE’s Blake Shaw tells CDM:

Just writing you to send our new video painting video which shows us video painting in London, Paris, Berlin, Belgrade, and Bristol. Since the last video we extended the software so that we can paint layers of video, allowing us to create live video collages. We even created a 5 meter telescopic IR paint roller so that we could do a two story tall video painting in Bristol.

More details:

New media art duo SWEATSHOPPE aka Blake Shaw and Bruno Levy are back from Europe with a new video that showcases their live interactive video wheatpaste in Berlin, Bristol, Belgrade, London and Paris. Over a two week period the duo pasted their videos in over 10 spots including the Berlin Wall, Les Invalides, Cordy House and even constructed a 5 meter telescopic electronic paint roller to create a two-story tall video painting in Bristol.

Video painting is a technology the duo developed that allows them to create the illusion that they are painting videos onto walls with electronic paint rollers they built. It works through custom software that they wrote that tracks the position of the paint rollers and projects video wherever they choose to paint, allowing them to explore the relationship between video, mark making and architecture and create live video collages in real time.

sweatshoppe.org
brunolevy.com
blakeindustries.com

Okay, great: you can project on facades and surfaces in ways that makes the image tailor-mapped to the surface. What else can you do?

Rafal Bielski and a small team from Poland provide a glimpse of a more awesome, more futuristic future. Here, projection mapping isn’t content with a still, static surface, like a building. The surface and the projection can both move, aided by robotic servos. As that image tracks the object, the combined project comes further to the dream of transforming the physical reality around us with digital visuals.

How cool is this? Well, for starters, that video above is not something done in post. It’s not a special effect. It’s live, real-time projection you’re watching – really. (No one hiding behind the curtain, either.)

Rafal explains that the result is “a combination of interactive real-time 3D projection, robotics and augmented reality.”:

MPS is the realization of an idea of pushing projection mapping one step forward, making it even more spectacular and involving. In short, we have synchronized rotation of stepped motor engine with “geometric” projection with respect to the position of the viewer. This is the moment where we get the spectator to “believe” that this is a metallic, weird box rotating in a front of the window of the skyscraper. It was shocking to me when I made a very first panel to show our friends that Vimeo flick. Most of them — like 80% — thought it was all post-production, a 3d animation (the industry three of us have come from) and Arek Rekita is just keyed-in there. So imagine the
sensation when you are actually standing in front of it. Its physical form there is real, and I think you can sense those qualities even through the computer’s screen.

The ability to visually influence human beings is weakening. We are getting more and more resistant to that form of communication. (When was the last time you heard heard that a movie had “good CG FC, really, go see it”?) In the film industry, I hope we are getting back to the era of a good script over spectacular effects, but in advertising, there is a whole new world coming in. I could rant about it for hours, so lets get back to MPS. The upper projection surface part is a 50 kg vinyl polymer structure, reaching 3.5 m in height. The base is a custom-made device from certified components, mainly stepper motor connected and controlled via Arduino.

The PC has twin GeForce cards driving three projectors – Benq W1100 and wide-lens Mitsubishi 230u-st. That worked well, but the system also worked with Christie Roadster monsters not in this video. Of course, we went for Derivative TouchDesigner as our interactive visual programming environment. It was a spectacular
learning process, adapting third-party libraries, synchronizing devices, and writing tools to quicken calibration processes. Yeah, we developed our own calibration software: it’s a semi-automatic little tool with live quality. We are going to introduce it on our Facebook profile soon. Stay tuned.

More is coming, too. Rafal tells CDM, “in the near future, we will presents our in-house software for semiautomatic projector/3d calibration and others.” Yes. We’ll look forward to that.

Consider yourself teased.

MPS project.
Realtime projection+servo+augmented reality.
Rafal Bielski : Programming, CG
Arek Rekita: Programming, Devices
Rafal Osmolski: Support

White Kanga is based in Warsaw.

Mapping projections to architecture, and connecting music to visual metaphors are nothing new. But many of these projects leave the control to performers; audience members simply stand back and watch. In a project for HP promoting their TouchSmart PCs, interactive artists ceded that control to participants. Instead of the computers being in the hands of the performers, they’re touchable by anyone, for an open, collaborative experience of the work.

The project makes use of a number of ingredients. The HP TouchSmart PC provides a big, touchable display, much larger (though less mobile) than a tablet like the iPad. On the software side, custom interfaces for the participants manipulate sound and visuals built in not one but two visual programming languages: Max/MSP and (Windows-only) TouchDesigner.

The project is the work of two firms, Waveplant and Leviathan. Joel Corelitz writes from the Waveplant side to share the work with CDM readers:

My company (Waveplant) collaborated with Leviathan to create an OSC-controlled projection mapping installation for HP to promote their TouchSmart line of PCs. Participants controlled the sound and visuals via the PCs with custom-created OSC interfaces (in TouchDesigner). Max/MSP was used to drive the audio and the visuals.

I was responsible for all the audio creation as well as the Max/MSP programming, and Leviathan created the visuals and handled the projection mapping.

Just to reiterate – Leviathan is the company that’s responsible (amongst many other things) for the visual component of Amon Tobin’s ISAM tour. My company / personal brand Waveplant was responsible for all the audio in this project as well as the maxMSP programmed back-end that was responsible for the live audio interactivity. Leviathan designed interfaces in TouchDesigner that transmit information to Max/MSP. Max/MSP would respond with audio in various ways while sending OSC information back to those interfaces and TouchDesigner which reflected those musical changes visually.

As an artist, I’m really excited about these kinds of marketing efforts. They’re incredible opportunities to push creative boundaries and subvert the more “linear” forms of advertising that we’re used to. What’s also really unique about them is the way they let the art stand on its own while at the same time involving the product in a hands-on (literally) capacity.

Here’s an extended documentary of the project, with requisite shots of Chicago’s famous L train. (For some of you, that should cause a Pavlovian hunger for deep dish pizza.)

More information:
http://www.waveplantstudios.com/HP-Play-Big
http://www.waveplantstudios.com/HP-Hear-There-Everywhere

Agency for this project:
designkitchen

It’s not just projection mapping on buildings as a cool gimmick any more. Live visuals increasingly means adapting to spaces, using multiple outputs and multiple projectors, tailoring your visuals to irregular surfaces and integrating them with environments. It’s projection, but it’s also dealing with LED walls. And flexible output isn’t just something expected of high-end shows and media servers: it’s something that increasingly is becoming something the lowly, laptop-in-a-backpack VJ does, too. It’s making your visual imagination look as good as possible when others see it.

All of this means it’s great news to see ArKaos, a long-time name in live visual and VJ software, move toward integrating mapping with their software. I got to spend some quality time with ArKaos at Pro Light + Sound, the companion event to Musikmesse, last week in Frankfurt. They showed us the new video mapping technology, available in public for the first time. (I got to discuss details of this technology in California in January with ArKaos at NAMM, but it wasn’t ready to show yet.)

We talk to ArKaos at top in our own video; below, here’s what their booth looked like with this software set up:

You’ve seen this kind of mapping convenience in software before, of course. The popular MadMapper tool, for one, offers similar features for taking live visual output and mapping it to three-dimensional surfaces. MadMapper works with any Syphon-compatible Mac application, too, whereas – for now, at least – ArKaos is only talking integration with their software.

But just as MadMapper’s approach has some advantages, ArKaos is bringing something new to the table. For one, Windows users aren’t left in the dark. ArKaos tells CDM that they’re using DirectX 9 extensions for high-performance routing between applications on non-fruit-related platforms. (Syphon is Mac-only, and uses OpenGL.)

ArKaos is also promising that by the end of the year, this functionality will be integrated directly with their software – not requiring, as MadMapper does, running a separate app. I also like ArKaos’ approach to output and masking: it’s a whole lot easier to see which surface is which and on which output, so multiple projectors (or LED walls) become convenient. Once integrated in something like GrandVJ, that could be even better.

As ArKaos describes it:

Designed for people on the run who need to setup mapping projects in very short times, the application can deform and project individual layers on the shapes of a mapped stage, allowing to easily make visuals flow around complex shapes in just a few clicks.

Right now, this is all an extension to MediaMaster, but if you can’t afford that high-end media server solution, it’s coming to GrandVJ, too.

ArKaos gives CDM the scoop on their planned timetable and release schedule. These are planned dates, not set-in-stone ship dates, but it gives you a good idea of what to expect:

May 1st

We release the mapper running with MediaMaster, the Video Mapper is V1.0 and MediaMaster is V 3.0 beta

MediaMaster 3.0 users needs the Pro license to have access to the Video Mapper

June 1st

We release the Video Mapper running with GrandVJ 2.0 beta

GrandVJ will cost 649 euros with Video Mapper (Pro), the upgrade for the mapper will cost 349 euros

July 1st

we release MediaMaster 3.0 final that can load mapping files and does not need the Video Mapper once the mapping is created.

August 1st

we release GrandVJ 2.0 final that can load mapping files and does not need the Video Mapper once the mapping is created.

Most of all, I’m glad to see the mapping field heat up. Competition can only be a good thing. I love the power of Syphon to route visual textures between apps, but having these features designer directly for a piece of software is also promising.

We’ll get our hands on this as soon as we can, natch:
http://www.arkaos.net/

Let us know if you have any questions for ArKaos.

Gallery

All images courtesy ArKaos, and since they naturally represent in-progress software, may not look exactly like the final product.

Lisbon, Portugal-based multimedia studio Dub Video Connection shares their 2011 showreel with CDM, as they play big-screen visuals with some of electronic music’s biggest acts. (Yes, it helps having this kind of showreel and documentation!) I love the high-energy abstraction and patterns amidst high-contrast ephemeral imagery, so I asked the artists to tell us a little about their inspiration and the rig that makes it happen. They tell CDM:

We are a multimedia studio that explores video art as a tool of improvisation, speech, and ambient design. We are based on a premise of pure visual impact, where the influences of musical method merge with the imagery of light, painting and cinema.

We use a quite-basic rig but we try to explore it at the top!

We use Modul8 to control the visuals and we mix it with two computers in order to improve performance. When we are working on video mapping, we use Max/MSP and Jitter to break [down] the mesh output. Nowadays, we are developing new content and exploring reactive and generative videos, integrating those in our shows.

All content is produced and developed by our team in our studio in Lisbon, Portugal. In our VJ reel, we use content from our VJ performances in festivals and clubs in Portugal, Spain, Italy, Germany, UK, Switzerland, Holland, and France.

We will be in Mannheim, Germany for Time Warp 2012 on March 31.

We are responsible for the stage design and all visual content. We are preparing a multimedia performance to present in Mannheim and then in Milan (October) and somewhere in Time Warp Holland in December 2012.

Thanks, Dub Video Connection people! We’ll be watching.

https://www.facebook.com/dubvideo

If you have a showreel you’d like featured, send as much quality documentation you can and be prepared to tell us a bit more about what you do, and get in touch!

For all the growth of three-dimensional mapping techniques, the surfaces are often (quite literally) dull. In Prismatica, an installation piece, the material is part of what makes the work so beautiful. Color and geometric form drip through the crystals like liquid, in a work that revolves like a modern, digital mandala, a kaleidoscopic meditation.

The artist, Melbourne, Australia-based Kit Webster, sends this work this way, and writes:

Prismatica consists of an arrangement of pyramid-shaped crystals affixed to an LCD screen and illuminated with programmed geometric animation. The animated patterns are precisely mapped to the vertices of the crystals, illuminating them individually and in formation. The animations are further refracted through the geometry of the crystals in accordance with the shifting perspective of the observer, which in turn alters the way the illuminations appear and interact with reflections of surrounding lights within the space.
This piece acts as an extension of the visual and perceptual experimentations of my immersive installations.

Soundtrack, of course, is the wonderful Alva Noto + Ryuichi Sakamoto collaboration, “Insen”

More: kitwebster.com

It’s also nice seeing this early sketch, via Kit’s Vimeo:

Visionary 3D scanning, computer vision, and digital media guru Kyle McDonald is back again with more tools that break down the boundary between the computer and the world. Kyle tells us he spent a great part of the fall in residence at the Yamaguchi Center for Arts and Media (YCAM) in Japan. He worked with experimental projector/camera rigs — and now we get to enjoy some fruits of those labors.

http://interlab.ycam.jp/en/projects/guestresearch/vol1

The software collection is free and open source (MIT License), built for OpenFrameworks, and already includes some documentation. (The ideal is contributing back to the core of OF, adding to this artist-savvy C++ coding tool.)

Mapamok

A new experimental projection mapping tool called Mapamok is the major highlight for me. As Kyle tells CDM, “the idea is to get people away from clicking on so many points and drawing so many masks. Instead, you load a 3D model of the scene and then click on a small number of points (8 to 12 points), and the whole projection is automatically calibrated.”

Automatic mapping and calibration? Sounds good to me.

The tool is completely free, as part of the ProCamKit. There’s both a download an an (English) tutorial guide:
http://github.com/YCAMInterlab/ProCamToolkit/downloads
http://github.com/YCAMInterlab/ProCamToolkit/wiki/mapamok-%28English%29

Video, at top, is described thusly:

Music: “Mass” by Vaetxh [Vimeo]
mapamok is an experimental projection mapping application that is part of the Projector Camera Toolkit developed during Guest Research Project v.1 at YCAM Interlab.
This video demonstrates a workflow for fast projection mapping involving: measuring and modelling the scene (5-30 minutes) followed by installing and calibrating the projector (5 minutes). Because mapamok is built with openFrameworks, everything is rendered in realtime. This allows for the technique to easily be extended to interactive applications. Near the end, an example of an interactive sound+visual mapping is shown.

But there’s more to this kit, cool:

3D Scanning Meets Projection

“The 3d scanning + projection stuff is something I hinted at being possible a long time ago, back when I got started with structured light,” Kyle explains. “Elliot Woods has also done some great work using the [Microsoft] Kinect for relighting.”

For more on 3D scanning and Kyle’s “structured light” concept – an effect we’ve seen in a number of music videos now – everything is at the Google Code site, from code to discussion:
http://code.google.com/p/structured-light/

Here’s what’s happening now. (Alas, poor Yorick, I knew him, a fellow of infinite jest, of most excellent fancy. Also, glad he wasn’t a dissected frog. That’s just cruel.)

Music: “Hoofbeat” by Dustmotes [SoundCloud]
This video demonstrates an experimental projection mapping application that is part of the Projector Camera Toolkit developed during Guest Research Project v.1 at YCAM Interlab.
First, the scene (the skull) is scanned using gray code structured light. This scan data is decoded into a 3D point cloud. This is possible because the camera and projector are calibrated and the position and orientation of each is known in advance. At :33 there is a brief demo of editing a GLSL shader in realtime, making a minor change that changes the width of the projected lines. Then the skull itself is shown again, with the projection mapped visuals.

shadowplay

“The shadowplay work, i’m pretty sure it hasn’t been done like this before,” says Kyle. “It’s the first steps towards a bigger project I have in mind.” It’s a bit tough to follow, but the idea is aligning projectors, then – using the white light created by the two projectors – manipulating shadow in a way that would normally be physically impossible. It’s clearly just a first step, but you can see some compelling potential here that could evolve along a different path than we’ve seen before.

Actually, that said, for me the best part of this is hearing some of Kyle’s music, which is also lovely. (Is there anything you don’t do in digital media, Kyle?)

Music: “Say It” by kylemcdonald [SoundCloud]
shadowplay shows an experimental interaction paradigm built with the Projector Camera Toolkit, developed during Guest Research Project v.1 at YCAM Interlab.
This video demonstrates some shadowplay experiments, built around the idea of using two projectors that are perfectly aligned. The projectors display the inverse images of each other, so that without interference the screen appears white. When an object enters the space, the structure hidden in the projection is revealed in the shadows. The technical challenge of aligning the projections and calibrating the colors of the projection is solved with the Projector Camera Toolkit, using structured light scanning and some novel feedback-based calibration techniques.

http://github.com/YCAMInterlab/ProCamToolkit

Rooted in pre-digital tradition, the turntables and mixer here get a different kind of digital upgrade: sound-responsive visuals transform the decks.

Built by Austin, Texas-based student Mark Morris, artist E.N.S., and collaborators, this work in progress demonstrates what happens if you point the projector directly at the DJ’s rig. Cleverly, they map the platters themselves as they rotate, as well as add interface elements from the DJ software and responsive audio waveforms. From Mark:

This is a projection mapping project we’ve been working on for about a month as of uploading.
It’s a user interface for Serato Scratch Live that is vinyl rotation-reactive, as well as sound-reactive, and makes it possible to mix using Serato without having to look at your computer screen whatsoever. We have real-time waveforms streaming across our mixers, or our vinyl platters, and we have started to incorporate video effects that are controlled by [Nintendo] Wii remotes. We are also working on controlling effects with motion capturing using a camera.
Much more to come!
The Song from the video was made by one of the creators of all the cool stuff you saw in the video: Mark Morris

Music from E.N.S. is available on Facebook.

Thanks to Francis Preve (Academik Records, Keyboard Magazine), who’s fortunate to have Mark as a student and points this our way.

We’ll have more as they get further along, but this is already looking great.

Update – via comments, here’s another instance of projection mapping onto decks, by MrGeeBee, a Luzern, Switzerland-based developer moonlighting as DJ. I wouldn’t be surprised if there are other examples, as well. Heck, I hope there are – go make more, folks.