motion distortion 2 from andrew benson on Vimeo.

Datamosh? (The “forbidden” but harmlessly meaningless word?) Video squishification? Mushy data?

Call it what you will, but applying real-time distortion and displacement to video so that video textures become flowing layers of pixels looks absolutely beautiful. Andrew Benson of Cycling ‘74 has only just begun playing with this in Jitter using GLSL shaders, and already the results are really compelling. (For a simpler example that looks more like the compression artifact technique we’ve seen recently, have a look at the second video – though, personally, I like the more sophisticated, layered approach of the video at top. This is going some very cool places.)

This is a Jitter patch, but would be simple enough to port to code for Processing, FreeFrameGL (which implements shader code), or other tools, too, in case you can’t bear being away from your moshness.

Andrew writes:


I recently posted a couple of Jitter patches and shaders that implement a really basic optical-flow based distortion effect (think realtime datamosh, among others) on the Jitter forums:
New Distortions [message #169387]

Super fun to play with. Another excuse to dance in front of your computer, or a way to convince others to do so.  PUSHING PIXELS.

A demo video too:

The idea came out of some R&D that I was doing for a current collaborative live video thing I’m working on, and the implementation from some random notes on websites scattered around the internets.  It all kinda happened by accident while I was distracted… Hope you enjoy.

I’m actually quite interested to see a performance comparison with OpenCV tools for the same technique – and, likewise, how computer vision routines in general can be warped to these sorts of aesthetic purposes. Thanks for this, Andrew, and if this inspires other folks to develop this more, I’d love to see the results / patches / code!

FAk30ut Stylezzzz!!!! from andrew benson on Vimeo.


How to Datamosh with Free Video Tools, “Datamosh” is the Wrong Word, David O’Reilly is Also Wrong

Data Moshing the Online Videos: My God, It’s Full of Glitch

  • Kyle McDonald

    An optimized repos! This is super helpful for a ton of motion-based processing. I wrote a patch a few years ago for Jitter emulating After Effect's time warp with motion estimation using optical flow + repos, but it had to run in non-real time. It looks like this runs a lot faster, allowing a VJ (or whoever) to plug in a camera to their setup, capture video of a performance, and slow down time without getting all stuttery going from frame to frame.

  • Peter Kirn

    Right, exactly.

    I understand there are also efforts to translate OpenCV — which is currently CPU-bound — to OpenCL for GPGPU acceleration (or, presumably, more parallel operation on future Intel processors).

  • Andrew Benson

    The nice part about doing this on the GPU is that the optical-flow/distortion processing is really efficient, so the bottleneck for speed is really the transfer of pixels to the GPU, not the effects themselves. Before anyone complains, I should also mention that this is a good-enough-for-experimental-video implementation of the Horn-Shuncke algorithm, and is not exactly perfect or complete. Looks cool though, right?

  • Neb

    Fantastic stuff. There is something seductive in the movement of the flow, very close to an effect I've been chasing a while myself. Is there any chance to see this ported to a QC shader, I'd love to toy around with it myself. In general, it's great to see more plastique approaches to video art.

  • vade

    So Andrew, can I release a Quartz Composer plugin for this now or what… ? :) :)

  • vade

    Neb: yeah, im on it :)

  • Andrew Benson

    I won't stand in anyone's way, just please let me know if you make something with it. I sent Vade this stuff awhile back in an earlier state. Maybe he can work it into a QC thing eventually.

  • Peter Nyboer

    I made a tutorial on how to incorporate Mr. Benson's bitchin-ness into our video software, DNA. Take a look!

  • Neb

    Vade: Fantastic, but aren't you supposed to be releasing a QC NES emu first? I want my glitched out kirby… ;)

  • Chris

    These are wonderful Andrew, love the organic fluidity. Can't wait to play with it as a Quartz patch. Thanks for sharing & I like your paintings too and how they relate to this coding/video work.

  • Andrew Benson

    There are some more cool demo movies here and here and all over my vimeo page. Lots of stuff you can do with this simple effect, when you combine other processes in the feedback chain or alter the distortion map in different ways, or combine with geometry.

  • Pingback: Create Digital Motion » Video Tutorial: Get Max-y Jitter-y Goodness in Cell DNA, for Moshing Your Optical Flow

  • Chris

    Thanks for the links Andrew, like the out of control swirling oil quality of the Splatches one. Lots of interesting possibilities.

  • memo

    hardcore! m/

  • Pingback: Digital Video Art: Two Interesting Techniques | Jim On Light

  • Pingback: Liquify Live Video « Ryan Pattie Interactivity Installation and Performance Blog