processing to MadMapper via Syphon [Test] from espadaysantacruz studio on Vimeo.

Wish you could get Modul8 to take an input from some other tool, so that in addition to playing videos, you could “insert” custom visuals as you play?

Wish you could VJ with Processing – without giving up your VJ tool, too?

Wish you could use Processing with MadMapper for custom project-mapped generative visuals? (I’m going to pause at the tiny handful of people who know what that question means. If you’re still with me — congrats.)

Wish you could do all of this with some other combination of tools on the Mac? (Hint: if it supports Syphon, you can.)

Media artist Miguel V. Espada answers just how to make all of this goodness a reality in an exquisite set of easy-to-follow, step-by-step instructions. Even the render-weary VJ could follow them. (Trust me – been there.)

In fact, I’m just about ready to go through every single one of the tutorials here. Kinect and Processing and Processing and Modul8 and Processing and projection mapping … Miguel, you’re our hero.

http://www.miguelespada.es/?cat=41 [thanks, Ilan!]

And all of these techniques promise to get better with time.

Hear that sound?

That’s the sound of visual sets getting done and people’s thesis work getting finished in time for spring and new, cool stuff happening the next time you go out and see visualists play. Mark my words. Start your engines.

Tutorial: Syphon input in Modul8 from instructions on Vimeo.

Vital public service announcement: Want to make sure this all continues to get better? Donate to the Syphon project. Despite the fact that this platform is utterly free to use for Mac users, and despite the fact that it has unleashed incalculable awesomeness on our community, I hear through the grapevine (or was that my HDMI cable?) that donations have been disappointing. Make that cease to be. For the price of just one coffee / Club Mate a day…

http://syphon.v002.info/

Processing has done wonders in popularizing for designers and artists the notion of using code to express visual ideas. Now, what if you didn’t have to fold the hinge of your laptop to explore ideas? What if it were as near as an iPhone or iPad? (Or, I suspect, other platforms, too?)

PR0C3551N6 isn’t the first tool to venture into these waters, but it looks like it could be the most sophisticated yet. Bring up your device’s touch keyboard (you’ll need some way to tap up that code), and start sketching. Through the power of Processing.js, you can run your creation, for creating generative artwork, interactive interfaces, and other visual ideas. To aid your coding process, the tool highlights syntax and includes integrated documentation. And you can use it even without an Internet connection – perfect for a distraction-free train ride.

You can even share creations, making this a mobile sketchpad that you can bring into your development environment on your “real” computer.

There are sacrifices in this system, of course. iOS doesn’t support Java, so full Processing functionality is out of the question. With that, you lose library support and 3D functionality, and console support is limited, though the print commands work. But that’s hardly a deal killer, because what you’d most want to do on a mobile platform – even a powerful tablet – is lightweight sketching, with the bigger, fancier projects on your laptop or desktop machine.

Ironically, the one mobile platform on which you can run full-blown Processing code is Android. But because Android apps require a sophisticated build toolchain on a desktop, I doubt we’ll see coding for Processing for Android directly on an Android device – at least, not without a nearby computer to do the heavy lifting. If you are a jealous Android user, though, this kind of functionality could be incorporated into a native Android app, too.

I hope developer Michael Markert releases the code under a free license. I’d happily pay more for the app on the App Store if I thought it’d help open development of the idea, and such a release could help assist people trying other platforms, including but not limited to Android. (Having a similar environment that worked across platforms has appeal, too.) But regardless, it’s admirable work.

More information:
http://processingjs.org/
http://processing.audiocommander.de/
Blog post: ANNOUNCING PR0C3551N6: PROCESSING FOR IOS [audiocommander.de]

I’ll post an update when it’s done and available! Thanks to the illustrious Ilan Katin for the tip!

Then: flat, rectangular displays. Now: projection mapping flat images to 3D objects. Next: volumetric displays.

From New York’s Chelsea neighborhood come two very inventive visual projects, opening in January.

Matt Parker, veteran of New York University’s ITP brain-building program (don’t ask what it stands for), leads research at NYC’s Eyebeam in some seriously-cool volumetric display tech. With the addition of Kinect, it takes on a human element. And an upcoming DIY kit means you’ll soon be able to build one of these yourself.

Lumarca is a truly volumetric display which allows viewers to see three dimensional images and motion. The system requires only a computer, a projector, and common materials found at most hardware stores. This provides an affordable platform for artists to design compelling content that conveys information, narrative, and aesthetic information in a new way. Lumarca is a collaboration between Albert Hwang, Matt Parker, and Elliot Woods.

Parker and Hwang are working now at the Eyebeam research center. Their work will open on January 12 in a new show there. Eyebeam’s Wiley Aker explains:

It’s a volumetric display, allowing visitors to see three-dimensional digital animations in real 3d space. The installation, made of a computer, a projector, and common materials found at most hardware stores, provides an affordable platform for artists to design compelling content that conveys 3d information and narrative in a new aesthetic way.

More information:
http://madparker.com/lumarca/

But wait – there’s more. Parker also has a Processing-powered piece called MahungArt up in the window in January.

And also part of the Jan 12 opening is GIFPUMPER, by resident Slava Balasanov, which Aker tells us is “basically a collaborative, real-time website for creating 3d, animated GIF mashups.” Of course it is. The image above gives you some taste of the resulting mayhem. (For now, invite required. Ask nicely.)

http://gifpumper.com/

I won’t be in New York on the 12th, but if you are, the opening runs at Eyebeam from 6-8 pm.

http://eyebeam.org/

More Lumarca, from SIGGRAPH Asia 2010, complete with music from our friend nullsleep:

Microsoft may be running showy ads that show imagined applications of 3D computer vision, but using technology like Microsoft’s own Kinect, hackers are making sci fi reality, right now. Art && Code, held in Pittsburgh, Pennsylvania, was an epic gathering of artists and hackers and hackers become artists and artists become hackers. It was an extraordinary convergence of learning and making, workshop and hacklab, technical brain dumps and creative experimentation. Special to CDM, Sofy Yuditskaya brings us a number of highlights. For her part, Sofy herself presented how to make the Kinect camera work with free software graphical multimedia environment Pd, something I hope we’ll look at more soon. Here’s what Sofy brought back from the conference, so you can see what people are doing and learn yourself. -PK

Last month, Carnegie Mellon University held its now annual conference/festival Art && Code — and this one was in 3D!

Lovingly organized by Golan Levin and a core group of administrators, the gathering created a supportive and productive space for artists, students, hackers, makers, researchers & educators, and corporate developers to have a conversation. Props to everyone involved.

I came in on Thursday at the end of the hackathon preceding the main event. The power of structured light scanning was part of Elliot Woods’ and Kyle McDonald’s Calibrating Projectors and Cameras workshop, documented on an extensive wiki:
4A: Calibrating Projectors and Cameras: Practical Tools

There was a very universal performance by Jason Levine using Kinect tracking, beatboxing and throat singing:
jasonlevine.ca

A media archeology foray into the mysteries of the Meréorama by Erkki Huhtamo was another highlight. Live piano accompaniment by Stephen L. I. Murphy incorporated some little-known and totally enchanting compositions made specifically for the installation by Henry Kowalski. (There’s a private video, available directly via the University of Chicago’s Film Studies Center site.)

Even Jonathan Minard’s documentation used a hack, shot with a rig set up for merging Kinect tracking and DSLR footage, as constructed by James George. [Photo above.]

The presentations were broadcast live, in anagraphic 3d as point cloud data set up by Joel Gethin Lewis, and other festival-as-labratory participants. [Photo above.] Ed.: I hear this wound up being a major focus of the hack sessions, getting this working – so creators, would love to talk to you more about this! -PK

Of those available online, here are my favorite speed presentations:

Kyle Machulis
FC 2009 Point Cloud

Using 3D scanning equipment originally meant for massive geographic scale (cities and larger) surveying to take a scan of 512 fursuits during the Further Confusion 2009 convention. Resulting data released as open source.

Toby Schachman
Time Travelers

Time Travelers is an interactive video mirror installation.

A monitor shows a still image. As the user approaches the piece, her three dimensional form emerges on the screen as a subtle distortion in the image. Upon further exploration, we discover that the image is a time-lapse video, and that the user’s distance from the piece is reflected as a time warp in the video.

Time Travelers uses the Kinect 3D camera to capture a depth image of the viewer. This depth data is mapped pixel-by-pixel to time on a source video. The closer the viewer is to the camera, the later in time is the video.

Time Travelers was created in openFrameworks. Source code is available on GitHub.

http://tobyschachman.com/

Asa Foster and Caitlin Rose Boyle
We Be Monsters

Created in Processing for Golan Levin’s Interactive Art & Computational Design 2011 Spring Semester course at Carnegie Mellon University

by Caitlin Rose Boyle :: www.sadsadkiddie.com
twitter.com/#!/rattusRose

& Asa Foster :: www.fosterthree.com
twitter.com/#!/foster_three

Music :: “Chocolate Milk” – The Two Man Gentleman Band
from “Dos Amigos, Una Fiesta!”, Serious Business Records, 2010

Christopher Coleman
w3fi

The W3FI is a social movement, a philosophy, a path to responsible connectivity between our online/offline lives and to each other.

Every day we find more and more of our lives integrated in the digital world, no longer able to lead separate lives, one virtual and one real. This means that we have to take control of and responsibility for how others see and relate to us in the digital world. We are proposing a new philosophy, a new strategy for our online interactions. It is called “W3FI,” a combination of WiFi, the word “we” and the slang use of the number 3 in place of the letter “e” to reference the digital parts of our lives. Currently people already consider WiFi to be a sort of invisible shared connection that is all around us and shifting it to “We” indicates a new awareness of how interconnected we really are online. The W3FI project is much more than an awareness campaign, it is a movement in social activism to ask a new set of questions for each of us every time we click, text, or share a photo.

http://digitalcoleman.com/#1689034/W3FI

Caitlin Morris
ofxOpenCV + Grasshopper

Frame differencing in openFrameworks, based on ofxOpenCv and ofxOsc add-on examples. Amount of motion from frame to frame is translated to aperture size in a basic Grasshopper model. Ed.: Heh: for the 99% of you who didn’t get that, watch the video! -PK

http://a.parsons.edu/~morrc963/canopy.html

Nick Hardeman
Follow This

Follow This! is a twitter based first person shooter game utilizing the Microsoft Kinect camera. The game gives you gatling gun arms that allows you to pulverize twitter birds.

Thanks for all the help from the OF community!

http://nickhardeman.com/460/follow-this/

Molmol Natura
Building a sculpture with Kinect IR structured light

with openFrameworks and meshlab
CNC routing
Ed.: Translation – a scan from the 3D real world is translated to digital and translated to the 3D real world. Replicators have arrived. -PK

http://www.mamanatura.tv/blog/?p=749
[warning: that Vimeo video autoplays, and the drill is loud Do you know how often "autoplay" is the right choice? Never amount of the time. -Ed.]

James George
DepthEditorDebug

In mid-2005, six weeks after the tragic subway and bus bombings in London, New York’s Metro Transit Authority (MTA) signed a contract with the high-tech defense and military technology giant Lockheed Martin. Lockheed Martin promised the MTA a high-tech surveillance system driven by computer vision and artificial intelligence systems. The security system turned out to be vaporware and the contract collapsed under lawsuits. As a result, thousands of security cameras in the New York subway stations sit unused.

It is in this technological atmosphere that we chose to collaborate. We soldered together an inverter and motorcycle batteries to run the laptop and Kinect sensor on the go. We attached a Canon 5D DSLR to the sensor and plugged it in to a laptop. The entire kit went into a backpack.

We spent an evening in the New York Union Square subway capturing high resolution stills and and archiving depth data of pedestrians. We wrote an openFrameworks application to combine the data, allowing us to place fragments of the two dimensional images into three dimensional space, navigate through the resulting environment and render the output.

http://www.jamesgeorge.org/works/deptheditordebug.html

Eric Mica
Überbeamer

The überbeamer is a means of making any surface writeable. It was created by Eric Mika as an ITP Thesis project in the Spring of 2011. Development is ongoing.

http://uberbeamer.com/

David Stolarsky

SwimBrowser is a freehand web browser, powered by Kinect.

The following features are implemented:
- Clicking hyperlinks
- Zooming in and out
- Panning
- Scrolling
- Choose text field
- Open new tab
- Open menu (e.g. favorites) and select
- Back/forward

swimbrowser.tumblr.com

Shawn Lawson
Operation Human Shield

Operation Human Shield is a re-humanization of Missile Command. The original game had players shoot down missiles that were on a trajectory to destroy the player’s cities or bases. Subsequent iterations of Missile Command have changed styles, thematic opponents, and attacking/retaliation technologies. My variation puts the player into the game and forces them to use their body to deflect/swat away at incoming missiles. The integration of the body references real-world use of civilian human shields. While war tactic is illegal by any nation who is party to the fourth Geneva Convention (approximately 195 countries), the practice of human shields is still in use today.

http://www.shawnlawson.com/#operationhumanshield.html

Shawn Sims
roboScan

roboScan is a 3D modeler + scanner that uses a Kinect mounted on a ABB4400 robot arm. Motion planning and RAPID code are produced in Robot Studio and Robot Master. This code sends movement commands and positions to the robot as well as the 3D position of the camera. C++ and openFrameworks are used to plot the depth data of Kinect in digital 3D space to produce an accurate model of the environment.

This work was done by Shawn Sims and Karl Willis as the first project of the research group Interactive Robotic Fabrication. This project was also presented in Golan Levin’s sp2011 course Interactive Art + Computational Design. The facilities of the Digital Fabrication Lab in the School of Architecture at Carnegie Mellon were used in the making of this project.

Karl Willis http://www.darcy.co.nz/
CMU-dFab http://cmu-dfab.org/
CoDe Lab http://code.arc.cmu.edu/
IACD sp2011 Golan Levin http://golancourses.net/2011spring/

http://sy-lab.net/#1102351/roboScan

http://artandcode.com/3d/

Thanks to Sofy for sharing this. If you were at Art && Code and want to share more of your own work, tips, code, art, or favorite moments from the event, get in touch.

Targeting game designers, educators, and students – but also clearly of interest to anyone who loves creative coding – Codify is a lightweight coding environment that allows you to work directly on the iPad. That is, you don’t have to use the full-blown iOS API, and you don’t have to connect a computer or fire up Xcode. You just start the app and begin coding, in a way that ought to be very friendly to users of Processing.

We’ve seen Processing.js running in a browser on a tablet, but this goes further. The environment is visual and interactive – there are even clever gimmicks, seen in the video, that make it easy to pull up a color picker or image asset browser right from the code editor. You’re typing code, but the Lua-based environment, inspired by Processing (among other tools), means a very small number of lines of code can do a lot.

Given a brief look at it, it’s also one of the nicest-looking code environments I think I’ve ever seen – neither overly spartan in its minimalism nor a copy of iTunes (cough, Xcode) nor the usual, overblown, overloaded IDE (cough, Eclipse). Features:

• In-line reference documentation
• Auto-completion
• Touch visual access to color, image, and numeric values
• Interactive code launching
• Full access to multi-touch, accelerometer

I’m confirming with them whether or not you can distribute tools built in Codify without Codify itself; I’m guessing you may not be able to do so. If you can’t, that would distinguish it from OpenFrameworks on iOS or Processing on Android, for instance, each of which are built for deployment. On the other hand, even if you can’t distribute your work directly, it looks like a brilliant prototyping tool. (The Processing for Android wiki provocatively suggests that someone could work out how to build a similar tool for Processing; I have no idea how practical that’d be, though, given the complexities of Android application packaging. But I should mention it, as Processing users will no doubt think of that, especially if they own an Android device.)

For now, enjoy some screenshots of the tool in action, as well as a game produced as an example:

http://twolivesleft.com/Codify/

Grab a Bluetooth keyboard and go to town. US$7.99 on the App Store right now.

One early video, hours from the release, shows a quick build of a particle system:

For reference, here’s a (free and open source) Processing.js IDE. Now, it’s nowhere near as fancy, but you get one significant advantage – because it’s based on the Web, you can run on iOS and Android and Chrome, just for starters.

Processing.js Mini-IDE for iPad, iPhone, Android, Chrome

Via a round-up on the excellent Digital Tools blog, Paragraf is a fun little GLSL editor.

And for more Lua-based game development fun, endorsed by the creator of Codify, check out LÖVE (works on Mac, Windows, and Ubuntu, if you’re not on the tablet bandwagon):
http://love2d.org/

I haven’t looked at the state of those projects recently, though; this could spur more of this kind of development, which I think would be terrific. And clearly, you can – at present, at least – do some different kinds of things in the Web environment and the “native” (Lua-based, here) environment, each with some of its own advantages. Either, I’d say, could make you want to quit your day job, find a nice, warm, shady grove, and sit and code under a tree all day on your iPad.

Generative physics animations in Illustrator most certainly fall into the “because it’s there” / “for no very good reason other than it’s possible” category. But the results are hilariously awesome. See above, Adobe fans.

The tool behind this is very cool, too. Processing lovers who don’t already know the insanely great toxiclibs library – and its ability to unlock complex math and geometry magic with very little effort – should absolutely go explore:
http://toxiclibs.org/ [links, documentation, and loads of terrific tutorials and demos]

It’s Java-ness makes it ideal for other Java-based projects, too, including Android. But that library, in turn, has come to JavaScript:
http://haptic-data.com/toxiclibsjs/

Scriptographer is a scripting plugin for Illustrator that opens it up to JavaScript – and may be a new reason to even use Illustrator.
http://scriptographer.org/about/

Add this easy-to-use JavaScript library to Scriptographer for Illustrator, and within that generative environment you get animated fun. See discussion:
http://scriptographer.org/forum/wish-list/toxiclibs/?pos=0#Post-4222

More animation in Illustrator: Ping Pong in Illustrator, also thanks to Scriptographer.

Brilliant. Anyone else come up with interesting stuff with this environment, we’d love to see it. Found via Andreas Köberle on Google+.

What began as a simple branding project for a pharmaceutical company evolved into beautiful, generative imagery, produced by a custom tool. Built with Processing (and the controlP5 library), the Actelion Imagery Wizard takes parameterized drawing to explode and reintegrate particle-driven illustrations of DNA, and faces, in what the designers call, neatly, “digital molecules.”

The project is the work of onformative, a design house uniquely dedicated to generative graphics.

Here’s what they say about the project (see also, from them, more documentation included here):

A new corporate design for Actelion Pharmaceuticals

For the rebranding of Actelion, a biopharmaceutical company, we developed a tool for automatic image generation that enables the generation of a unique, in-itself homogeneous graphic image world out of heterogeneous visual material.

“Create a new identity for our brand Actelion. Don’t touch the logo, but create something the world of pharma has not seen before.” Based on the idea “From medical industry to medical magic,” a new image was developed that rendered medicine’s invisible magical moment visible. Here, the new imagery is based on the smallest possible unit: digital molecules. These capture the magical moment when new medicine arises from molecules; innovative design for a company that supplies the world with innovations. We worked closely together with Interbrand in creating the graphic imagery and developed a software, that enables the targeted creation of graphic images based on photographs, whose visual information contained in the tool is translated into graphic elements such as points, lines and curves using special algorithms.

Thanks to magician and technologist Marco Tempest for the tip!

http://www.onformative.com/work/actelion-imagery-wizard/

Casey Reas, co-creator (with Ben Fry) of the elegant Processing coding environment so beloved now by artists, talks to our friends at The Creators Project in a recent video. Processing may be best known as only a tool, but it’s also become an open source conduit for ideas in aesthetics and digital art. Casey’s impact on its design is profound: the way the tool works, the way the code editor works, and even code examples are all partly his contributions (along with other collaborating artists). And when some of his personal work is a showcase for the tool, that’s doubly true. It seems to me akin to the development of artistic media at some of the peaks of technique – take, for instance, key moments in the evolution of painting. Tools and techniques, aesthetics, culture, and output are shared by practitioners all at once.

Casey also says something really beautiful about using color values in the way that you’d use sound in film, underlying emotion.

It’s a much-watch video, and Casey, I hope we get to talk a bit soon, as well.

Great video and documentation from comes our way from Eric Medine:

Documentation of generative visuals applications I built using the Processing platform, ported to the Android OS and run on an HTC Evo. Output is 1080×720 over HDMI to a Vizio LCD display.

Although the EVO has an HDMI output for video, the drivers are crippleware– designed to only display media “approved” by Sprint and its affiliates. Luckily, Sam Bero has been developing a custom HDMI kernel to allow mirroring of any kind of output, and I was lucky enough to be on the beta tester team. Thanks Sam!

Here’s a tutorial on how to port Processing sketches to Android and rooting/installing Full HDMI to your EVO: ericmedine.com/​tute_mobileProcessing

Fortunately, this won’t be the case with Honeycomb Android tablets running Processing – we’ve heard people with great luck on XOOM and the like. I also hear lots of good luck with iOS, a topic I hope we’ll cover soon. But it’s nice to see a hack make something work on a phone – impressive. Now we need more tiny projectors with HDMI in.

“Fine,” I hear you say. “But what would really make my day is an unrelated Wiimote-controlled video mixer.”

Your wish – granted!

Clear all the tablet hype away, and consider this: if someone told you a few short years ago that you could soon have a flat computer with HD screen and HD output, multi-touch interface, two cameras, and real OpenGL graphics, you would probably have fallen out of your chair. If you then said it’d also run Processing and OpenFrameworks sketches, some nerds might have actually passed out. And now it’s here. Most of the debates in tech circles center on, like, normal people, not people who fall in love with computing platforms for their artistic and psychedelic potential. Shame, that.

So, in that spirit, let me be the bearer of the good news from Google’s IO conference. Here’s the news that really matters:

Android now supports USB hardware. As I wrote yesterday on Music, Android now finally adds hardware support. On all 2.x and 3.x devices, this includes USB accessories which can be prototyped with Arduino. On all 3.x tablets – and possibly some phones, too, though I can’t confirm that – you also get proper USB host mode, which means the ability to plug in joysticks, gamepads, and even custom USB hardware. Best of all, none of this requires nasty legal agreements or NDAs or approval. (Gee… it’s almost as if you own the hundreds of dollars of hardware you buy.)

One specific item that I learned later, though –

You can use external cameras. When I first read this, I thought we’d actually be able to use external video cameras, which would be terrific and computer-like. I was wrong. In the meantime, though, you can at least pull photos and videos in a standard way; that’s good, at least.

Renderscript is pretty awesome. It’s not really news, but it was nice to see Android Honeycomb’s Renderscript engine, which lets you do some pretty sophisticated animation and 3D using very small amounts of code. The Android Developers’ Blog covered it back in February, but with renewed focus on Honeycomb in sessions (and the likelihood all of this will be on phones by the end of the year), it’s getting featured more prominently. See demo:

Wireless features. Less covered in the press, but 3.1 will add RTP streaming content transport, and – critical for things like touch OSC controls over wifi – more robust wifi lock and wifi performance, something that’s been pretty dodgy on these kinds of devices both on the Android and Apple sides. (Well, it will be if it works as advertised.)

Processing! Processing didn’t appear in Google’s presentation, but I can say that all this tablet stuff is very good for those of us wanting to play with Processing (or OpenFrameworks, for that matter) on Android. I’ve already heard really good things about Processing tests on the Xoom.

The vision: two tablets and a projector replaces two turntables and a microphone for the live visualist. Or, for that matter, lots of projectors and tablets. The mind reels…

Anyway, such is a preview of the state of things on the Android side of the fence. For more on iOS and HD output, I hope to have an update in a few days. Consider that a teaser.

Also, side note: we also learned Google TV isn’t dead, and we should finally see development – a la Honeycomb – this year. I randomly won a Logitech Google TV some weeks ago, so I’ll report back for anyone interested.

Android 3.1 Platform Highlights