Composer/artist Ryoji Ikeda has been on a roll lately, mounting massive immersive audiovisual experiences. His aesthetic talent, in crisp blacks and whites and SONAR-like pings above oceans of glitching noise, is in spinning operettas of data. The latest project, which closed at the start of this month at Berlin’s Kraftwerk, pulled apart the data from a car. As brand integration, it’s not quite a winner for subtlety, literally working from the components of a Honda Civic, but Ikeda’s unique stamp carries the day. His work is almost a showcase for contrast ratio, and sure enough top-notch projectors make that light and dark necessarily bold. (Credit to projectiondesign F32s, at 7500:1 contrast and 8000 lumens, reports creativeapplications.)

One of my fondest art memories of last year was spending time at Ikeda’s Transfinite at the Park Avenue Armory in Manhattan. This single show helped cement the Armory’s place on the map for installations and contemporary art, not just its traditional art shows. Its success was in creating an environment that felt oddly like a public park, enclosed indoors – very much not the thing you’d expect from a digital art installation. I saw couples making out, children running around and dancing across projected patterns, young people staring into space in quiet meditation, lying aside friends as if at a picnic. It had the feeling of the modernist plaza, a space that in its abstract severity seemed to produce an open container for whatever people chose. So, it’s worth revisiting those works, too.

Whatever the work, though, Ikeda’s designs are demonstrating real staying power, in a fusion of visual and sonic aesthetic sense, in digital punctuation.

Transfinite, 2011:

And more on data.anatomy, from Designboom:

Designboom walks through each visual sequence in detail in a story on their site

Pick up that TV and rock it, baby.

While recalling a now-obsolete technology and the work of artists like Nam June Paik, de/Rastra is something of a (delightful) lie. In the form of a television, it appears to be a self-contained, vintage instrument. In reality, it’s a simulation, a CRT with “altered anatomy” that uses a computer to drive faux vintage cathode ray visualizations and to produce digital sound. But the synthesis of visuals with the body of a television is wonderful, a play on past and present technology that produces an impossible electronic now.

The new soul of this ghostly machine is an Atmel 328 chip connected to various force and position sensors and buttons, communicating with a computer running Max/MSP via xBee wireless.

Like the repurposed art we saw last week, the obsolete “zombie” technology is reanimated. It’s a hack – those hacks are even freely documented – but one that produces a new function, even if the body of the tech becomes as decorative as it is essential. As the artist describes it:

The de/Rastra oscillographic synthesizer is a real-time audio/video instrument and computer-interfacing device that allows a performer to generate visualizations intrinsic tocathode ray tube technology while simultaneously creating the acoustic analog of the displayed imagery. By way of building, bending and mutilating, de/Rastra shows the effects of altering the anatomical makeup of a CRT television, revealing the intrinsically hidden potentials of the technology through the repurposing and restructuring of its own ability. Through hacking and exploiting the capabilities intrinsic to all CRT devices, the technology becomes repurposed as a performative interface, breaking down the device’s ‘consumption only’ nature. The performer is given control over the technology by removing it from the intended application and forcing it into an active state through a combination of physical and mental effort. The de/Rastra oscillographic synthesizer is an open source project and will eventually be accompanied by tutorials on methods of CRT hacking. Related tutorials can be found at http://www.crackedraytube.com/textstutorials.html but specific information regarding de/Rastra are in progress.

Can’t get enough ray tubes? Here’s more from Kyle – Cracked Ray Tube: a collaborative performance project by James Connolly and Kyle Evans.

Full details and technical specs, plus a paper on the project, at the artist’s site:
http://yaktronix.com/derastra.html

Thanks to Martin Backes for the heads-up on this one.

The folks at Teenage Engineering have teamed up with another Swedish-bred design house, IKEA, to construct about the simplest digital camera imaginable. I love their cheeky suggestions of features. This being Motion, one has to wonder – what would the simplest possible video camera look like? Jesper Kouthoofd, designer at TE, deserves credit for this one (and a sense of humor).

Worlds cheapest digital camera made out of one piece folded cardboard, a single circuit board for all electronics, camera sensor and integrated USB connector. The cardboard enclosure is secured by two plastic screws. The KNÄPPA camera was invented and designed by Teenage Engineering for IKEA. Teenage Engineering worked very closely with IKEA to make this project become a reality.

Via: IKEA KNÄPPA [now.teenageengineering.com]

That appears to be a Swedish word that means, in essence, “click” – as in a shutter.

With multi-touch fully exploited and the basics of camera vision largely understood, interaction moves to the realm of free space, “augmenting” your world with gestures that find some physical connection. They surprise by working in some way that seems intuitive and natural, somewhere away from what seems to be the realm of the computer. And early in the month of May, we see a flurry of new research in just this area. Not one but two projects from Microsoft hold potential, and one from MIT Media Lab is just … absurdly cool.

A summary:

• “Levitated Interaction Element,” out of the MIT Media Lab, uses a magnetically-floating ball as an interface. Advantage: free-form 3D interaction with physical, tangible feedback.
• “SoundWave,” from Microsoft Research and the University of Washington, uses an ordinary mic and speaker, with the aid of the Doppler Effect, to add natural interaction without additional sensors. Advantage: no special hardware needed for gestures – not even the use of a camera. Another win for the power of sound. (You can even listen to music at the same time.
• “MirageTable” from Microsoft Research builds on the growing power of Kinect by adding natural perspective and the illusion of real-world manipulation. Advantage: a more complete illusion. (though why aren’t they using back projection in the demo?)

More details. From top, “ZeroN” / “Levitated Interaction Element:

What if materials could defy gravity, so that we could leave them suspended in mid-air? ZeroN is a physical and digital interaction element that floats and moves in space by computer-controlled magnetic levitation.

http://media.mit.edu/~jinhalee/zeron

Title: Levitated Interaction Element
Researchers: Jinha Lee, MIT Media Lab Tangible Media Group,
in collaboration with
Rehmi Post, MIT Center for Bits and Atoms
Advisor: Hiroshi Ishii, MIT Media Lab Tangible Media Group
Institution: MIT

Gestures are becoming an increasingly popular means of interacting with computers. However, it is still relatively costly to deploy robust gesture-recognition sensors in existing mobile platforms. SoundWave is a real-time sensing technique that leverages a speaker and a microphone to robustly sense in-air gestures and motion around a device. It is capable of detecting a variety of gestures, and can directly control existing applications without requiring a user to wear any special sensors.

Researchers: Sidhant Gupta, Dan Morris, Shwetak Patel, Desney Tan
Institutions: Microsoft Research; Ubicomp Lab, University of Washington

In MirageTable, a 3-D stereoscopic projector projects content directly on top of the curved screen. The information is captured by the Kinect camera, which also tracks the user’s gaze. This enables presentation of correct perspective use to a single user on top of the dynamic changing geometry of the real world.

Researchers: Hrvoje Benko, Ricardo Jota, Andrew D. Wilson
Institution: Microsoft Research
Presented at ACM SIGCHI 2012

By the way, speaking as an immigrant in Germany, as an American citizen, and as a journalist covering international technological innovation, I’m always struck by reading the names in projects like this. Innovation is an international affair, and part of the strength of academic institutions (MIT, University of Washington) and corporate research (Microsoft) alike is its ability to pull talent from around the world to work together. Those benefits can produce wide-reaching opportunities for everyone – and everywhere – involved. Without knowing the specific background of these particular researchers, but based on what I see with the people I do know, I’d point to this:

Why America Needs More Immigrants [Wired Science; Jonah Lehrer writing last year for Wired and The Wall Street Journal]

It includes numerous factoids like this one: “In recent years, immigrant inventors have contributed to more than a quarter of all U.S. global patent applications.”

Worth considering as governments around the world debate immigration policy, freelancer policies, and other key decisions.

In particular, it seems that the many pieces for improving interaction design and natural interaction technology lie scattered around the planet, a bit like a Triforce of futuristic interaction. Now, of course, “patents” are one potential obstacle – the involvement of both corporations and academia in the patent system threatens just this sort of international exchange. But the progress of the ideas underlying those specific patents remains something that flows globally across a range of institutions.

And that pace of innovation can really make the Next Big Thing seem to emerge out of thin air.

We remember Adam Yauch today on Create Digital Music, with the help of Network Awesome. It’s doubly appropriate to talk video, as MCA was also the Beastie Boys’ filmmaker.

Our friend Benton-C Bainbridge did live visuals with Beastie Boys. He remembers Adam by sharing, at my request, some of his visual work with the band.

At top:

Beastie Boys tribute on VH1 Hip Hop Honors, using thermal cam, oscilloscope, vdmx, rescan and modded MX30 mixer.

And two more:

Beastie Boys on MTV VMA Latin America, using rescans, TiVo digital delay, modded MX30 mixer. the Matrix-style ‘bullet time’ camera rig is by Stuart White.

Beastie Boys on MTV Networks, excerpts from several shows includes the first gig I did with Beastie Boys with my dad’s Heathkit oscilloscope.

The Beastie Boys wanted a “live oscilloscope jam,” recalls Benton.

It’s beautiful work, and another way to see this incredible trio as we remember all MCA brought to it.

Benton is working on an installation opening this week in Nashville, Tennessee. He memorializes the collaboration with MCA on VHS tape in that low-fidelity set of art pieces, and recalls what it was like to work with MCA and the Beasties – including the reflection that MCA continued to work on his film projects until quite recently.

Benton-C Bainbridge on His Work With the Beastie Boys and Super Long Play!, Opening Tomorrow [Nashville Scene]

He tells Nashville Scene:

Adam was quiet and strong; he pushed hard for what he believed in. We’ve lost an innovator, a great American artist and a righteous human being. I’m sad, but I’m also inspired that Adam stayed with us for so long after he was diagnosed, and he gave us so much in his last few years.

This past weekend I’ve been thinking about how grateful I am to have worked with Adam. To get the job with the Beastie Boys, I showed him what I could do with their new (at the time) single “Ch-Check It Out” — run through an oscilloscope to make squiggly patterns that bounce to the tune.

For the last tape in Super Long Play! I re-create the oscilloscope tricks to “Ch-Check It Out” and then speak on camera about memories of Adam. Albeit with some audio hum — those are the perils of low-fi!

Benton’s Glowing Pictures is based in New York.
http://glowingpictures.com/

In moments that would have pleased Belgian surrealist René Magritte, an ordinary paint roller transforms a litany of European landmarks with video “paint.” Roll on, and it’s as though someone has punched a hole to another dimension. This isn’t a post-production effect: it’s all live, via computer vision and projection mapping.

The project is the latest work by international duo SWEATSHOPPE. We’ve seen them twice before:
Painting with Video: Live Projected Wheatpaste with SWEATSHOPPE
Hypnotic Colored Geometries and Balloon AV from SWEATSHOPPE

SWEATSHOPPE’s Blake Shaw tells CDM:

Just writing you to send our new video painting video which shows us video painting in London, Paris, Berlin, Belgrade, and Bristol. Since the last video we extended the software so that we can paint layers of video, allowing us to create live video collages. We even created a 5 meter telescopic IR paint roller so that we could do a two story tall video painting in Bristol.

More details:

New media art duo SWEATSHOPPE aka Blake Shaw and Bruno Levy are back from Europe with a new video that showcases their live interactive video wheatpaste in Berlin, Bristol, Belgrade, London and Paris. Over a two week period the duo pasted their videos in over 10 spots including the Berlin Wall, Les Invalides, Cordy House and even constructed a 5 meter telescopic electronic paint roller to create a two-story tall video painting in Bristol.

Video painting is a technology the duo developed that allows them to create the illusion that they are painting videos onto walls with electronic paint rollers they built. It works through custom software that they wrote that tracks the position of the paint rollers and projects video wherever they choose to paint, allowing them to explore the relationship between video, mark making and architecture and create live video collages in real time.

sweatshoppe.org
brunolevy.com
blakeindustries.com

Today on Create Digital Music, we examine DeFunct/ReFunct, the latest installment of a touring (Ireland, France, Germany) collective working with repurposed, rescued refuse technology:

Art From Trash, as ReFunct Media Makes a Symphony from Obsolete Gear [Videos]

The group works both in sounds and image, so here I’ll echo what I say there in visual form. (I feel, far from conflicted, that I’ve done my job if I find things that fit into both sites.)

Benjamin Galoun [aka Recyclism] makes work that, of all of these, is perhaps the most elemental. It’s just voltage applied to a screen, but in the mechanical preparation resembles nothing so nearly as a music box, “playing” generative pixels.

AbstracTris is a LoTech generative pixel art device. The pixels are directly controlled by applying voltage to the side pins of the GameBoy LCD screen.

Other works, too, played with low fidelity, downsampling and upcycling, direct application of voltage and circuit bending, and the creative misuse of gear, from Gijs Gieskes’ audiovisual instruments to rescued tubes doing wonderfully-glitchy things. Images:

From Gijs Gieskes, the Hypnotoad and a video sampler (seen, from top):

/ HSS3 Hypnotoad
The Hypnotoad is a character from the television series Futurama, it hypnotizes
everyone that looks at it, by generating a drone sound and wobbeling it’s eyes.
A Youtube user made a video loop of this Hypnotoad, that is about 10 minutes long. The HSS3 Hypnotoad is a hardware version of this Youtube video, that can run forever.

/ GVS1b
The GVS1b is a video sampler, that can be used to sample small clips of composite video and play them back in a grayscale depth of 1.5 bit.
In the exhibition setup there is a viewfinder used as a display, and a small security camera that films the person operating the GVS1b so the person can sample itself.

ReFunct Media 2.0
Karl Klomp / Benjamin Gaulon / Gijs Gieskes

In the “Practice of Everyday Life” Michel de Certeau investigates the ways in which users-commonly assumed to be passive and guided by established rules-operate. He asserts: “This goal will be achieved if everyday practices, “ways of operating” or doing things, no longer appear as merely obscure background of social activity, and if a body of theoretical questions, methods, categories, and perspectives, by penetrating this obscurity, make it possible to articulate them.”
“ReFunct Media” is a multimedia installation that (re)uses numerous “obsolete” electronic devices (digital and analogue media players and receivers). Those devices are hacked, misused and combined into a large and complex chain of elements. To use an ecological analogy they “interact” in different symbiotic relationships such as mutualism, parasitism and commensalism.

Voluntarily complex and unstable, “ReFunct Media” isn’t proposing answers to the questions raised by e-waste, planned obsolescence and sustainable design strategies. Rather, as an installation it experiments and explores unchallenged possibilities of ‘obsolete’ electronic and digital media technologies and our relationship with technologies and consumption.

Electronic Instant Camera by Niklas Roy

Electronic Instant Camera is a combination of an analog b/w videocamera and a thermal receipt printer. The device is something in between a Polaroid camera and a digital camera. The camera doesn’t store the pictures on film or digital medium, but prints a photo directly on a roll of cheap receipt paper while it is taking it. As this all happens very slow, people have to stay still for about three minutes until a full portrait photo is taken.

Photos: LEAP gallery Berlin, Trevor Good

It’s well worth reading the program notes for ReFunct, as it includes some lovely essays on glitch and aesthetics:
Benjamin Gaulon: “Hardware hacking and recycling strategies in an age of technological obsolescence”
Garnet Hertz & Jussi Parikka: “Five Principles of Zombie Media”
Rosa Menkman: “The moment(um) of void”
Alessandro Ludovico: “Circuit Bending: Repurposing The Past”
Eduardo Navas: “The Aesthetics Of Representation In Circuit Bending”
Phillip Stearns: “Error, Noise, Glitch: The Art of the Algorithmic Unconscious”
Defunct/ReFunct Catalog [PDF]

Piccolo is like candy for people who like robots and drawing machines. It’s affordable, cute, and downright fun. It’s going to be on your Christmas in July wanted list. How awesome is it? Let us count the ways:

• It’s under US$70.
• It’s Arduino-compatible.
• It works with Processing – ideal for drawing experiments.
• It’s open source hardware.
• It’s easy to assemble.
• You can make 2D and 3D things – and it appears to excel as a drawing machine.
• Connect it to Arduino, and you can make your own sensor-driven drawings and other creations.

I’m really excited to meet the creators when they speak at the Open Design Symposium in Linz, Austria, and present this workshop at LiWoLi afterward.
http://www.open-design.at/
http://www.liwoli.at/

I’ve signed up for their workshop, so hopefully will be building and testing one of these myself. I’ll let you know how it goes. I’m also speaking and performing at the event, talking about Open Design and music, and, of course, our MeeBlip. (I hope I can find some way of connecting MeeBlip and Piccolo, too. Wish me luck.)

Piccolo is the creation of the brilliantly-named Diatom Studio, in collaboration with Cheng Xu and Huaishu Peng from the CoDe Lab.
More: http://www.piccolo.cc/

“Natural interaction” is the phrase commonly applied to gestural interfaces. But a gesture is only “natural” once you’ve learned it. And as everyone from interaction designers to game makers have discovered, that can leave users confused about just what gesture they’re supposed to make. (Ironically, the maligned conventional game controller doesn’t suffer so much from this problem, as its buttons and joysticks and directional pads all constrain movement to a limited gestural vocabulary.)

A team of researchers has just shared a new approach to the problem. Coming from Microsoft Research and the University of Illinois’ Computer Science department, authors Rajinder Sodhi, Hrvoje Benko, and Andrew D. Wilson use Kinect not only to process gestures, but teach them. Here’s their description:

LightGuide is a system that explores a new approach to gesture guidance where we project guidance hints directly on a user’s body. These projected hints guide the user in completing the desired motion with their body part which is particularly useful for performing movements that require accuracy and proper technique, such as during exercise or physical therapy. Our proof-of-concept implementation consists of a single low-cost depth camera and projector and we present four novel interaction techniques that are focused on guiding a user’s hand in mid-air. Our visualizations are designed to incorporate both feedback and feedforward cues to help guide users through a range of movements. We quantify the performance of LightGuide in a user study comparing each of our on-body visualizations to hand animation videos on a computer display in both time and accuracy. Exceeding our expectations, participants performed movements with an average error of 21.6mm, nearly 85% more accurately than when guided by video.

The complete research paper is published online:
http://rsodhi.com/LightGuide.pdf

Thanks to co-author Rajinder Sodhi for the tip!

As ubiquitous in our lives as the digital landscape of sites like Google Maps can be, they’re in some sense private space. They strip our world of human beings (or freeze them in strange, invasive shots taken by roving vans), and put that space in the exclusive hands of private publishers. (Or, at least, one beginning with the letter “g.”) That has helped the groundswell of interest in OpenStreetMaps as an alternative. But looking deeper, we’re reminded of our role in physical and imagined space and what mapping itself can mean.

A film by Roel Wouters takes a creative approach to reimagining Google Maps, as crowds re-populate the area around Eindhoven’s Strijp S (the former industrial park in the Dutch city). Here, using overhead videography via fishing rods and tracking with orange traffic cones (“pylons”), the tiles in the map come to life with human movement. Crowds evoke the markers and overlays with fanciful movements in herds and miniature dances, drawing and scrawling and dancing across the surface of the map. The film calls pierces the veil between us and this constant, co-present virtual map of our world. It plays with the idea of maps as public space, and toys with the overhead perspective that soon becomes part of our self-conceptualization of our world.

It is a short film, a dance (complete with choreographer), a music video (complete with evocative, reflective score), an MGM color spectacular for the digital age.

Wouters may or may not have intended the political program, but the film for me re-humanizes the map. There’s no great arc to the film, no moment of revelation; it can become caught in its own technological ingenuity. But it still may have you seeing maps in different ways, and it demonstrates that tracking technology can be applied to radically different purposes in filmmaking.

(Really, just a few cameras, orange cones, and fishing line – yes, you can do things like this.)

More details:

What if Google Maps went live…
was recorded using six cameras attached to fishing rods on the former Philips industrial site Strijp S in Eindhoven during flux/S SQUARE ONE, the Prelude September 3-4, 2011

The crew with their fishing rod cameras:
http://www.flickr.com/photos/roelwouters/6309115250/in/set-72157627740231879

We used two thousand pylons as tracking markers:
http://www.flickr.com/photos/roelwouters/6309111936/in/set-72157627740231879/

Check out more making-of photos here:
http://www.flickr.com/photos/roelwouters/sets/72157627740231879/with/6232198782/ [slideshow embedded below]

Concept: Moniker commissioned by flux/S
Producer: Jan Schuijren – flux/S
Director: Roel Wouters – http://studiomoniker.com

Executive Producer: Vargo Bawits
On-Site Production: Marleen Hartjes
Communication: Maaike Delemarre
Pre-production: Chantal Kleinmeulman

Production Assistance: Eva Bullens, Eva van der Moer,
Walter van Beek, Jeanine des Bouvrie,
Henk van den Brink, Piet van Benthem

Photographer: Marc Faasse
Director of Photography: Sal Kroonenberg
Floor Manager: Eddy Stub
Camera Operators:Nicholas Burrough, Jeroen Simons, Edo Kuipers, Thijn Teeuwissen, Edwin Donders
Data Handler: Johan Bosgraaf

Choreography: Katja Heitmann
Composer: Merlijn Twaalfhoven
Sound Recording:Friso Pas, Willem de Wijs, Peter van Suyderhoud
Audio Post-Production: Peter van Suyderhoud
Metronome sound: Rik Elstgeest
Post-Production: Shosho
Soundtrack Title Sequence: Guru by Light Light

Special Appearances
Björn Ottenheim & Daan Schinkel – zZz
Luna Maurer / Moniker – Sand Pen
Karin de Wit – Hoola Hoop instructor
Roger Martinez – soundscape
Bas van Duren – speaker
Koninklijke Harmonie Phileutonia
Eindhovens Seniorenorkest
Stratums Muziekkorps St. Cecilia
Sambaband Batedeira
Escola de Samba os Malandros
Muziekvereniging Eindhoven West

Found via Rosa Menkman.

Behind the scenes images: