Multi-display visual installation by The Pixel Addicts for the London club space Matter.

Part of the big idea behind Resolume Avenue 3 is putting audio and video in the same real-time performance app, without sacrificing quality or live processing in either. One key ingredient, hope the developers, is a new codec that transfers video decoding to the GPU without taxing the resources the GPU needs for accelerated effects. For lower-resolution footage, you may not notice a difference, but with a modern graphics card, Resolume claim intense multi-layered performance at higher HD resolutions. We’re keen to see just how that stacks up in benchmarks, not only Resolume vs. Resolume, but Resolume’s DXV up against codecs like Photo JPEG in other apps. But before we get there, we first wanted the inside scoop from the developers.

I got to speak to Resolume’s Bart van der Ploeg. I knew Bart had been dying to show us the new codec for months, so here I get to grill him about questions a lot of us have.

The good news: no, this won’t screw up your GPU-based shaders.

How did DXV come about?

While doing our research for a new GPU-based rendering engine for Resolume back in 2006, we discovered that a large bottleneck for a fast GPU video engine was the limited bandwidth to the video card. The rendering of video frames with fancy effects at a high resolution was absolutely no problem for the GPU, but half of the time it was just waiting for all the video frames to arrive to the video card’s VRAM. We just could not get the video frames to the video card fast enough to feed the GPU to its maximum potential.

We reduced the amount of data that has to travel to the video card by feeding it compressed video frames instead of uncompressed frames that come from any other traditional codec. This makes the whole engine run much faster and reduces the CPU usage so we can use that for other things like audio effects.

We choose QuickTime because it’s cross-platform, but QuickTime .mov files only act as a container from which Resolume 3 reads compressed frames. This is why DXV videos do not suffer from the generally lower performance of Quicktime on Windows. Large parts of QuickTime are simply bypassed during playback in Resolume.

You note that it was developed with UnitedVisualArtists; what was the relationship between UVA and Resolume?

We hired Dave Green from The Pixel Addicts to help us with Resolume 3 part-time while he was also freelancing for UVA. UVA liked the GPU video decompression algorithms we had developed with Dave, so we decided to share the development costs to turn the developed algorithms into an easy-to-use codec.

The Pixel Addicts and UVA both support DXV in their high-end media servers, so it’s currently used for large shows in club Matter in London by the Pixel Addicts and on the Kylie Minogue tour by UVA.

One drawback of DXV, I understand, is that it does take longer to encode — thus, adds a little work to prepping your footage — correct?

Yes, encoding unfortunately takes longer than most other codecs; we are researching a solution for this.

In most cases, the file size is also larger than MJPEG (Motion JPEG), but in some cases
it’s smaller, especially when a video has large, simply-colored parts.

But reading larger files is not a problem, hard-drives are more than capable of reading the data fast enough. It’s the CPU that’s the bottle neck; the GPU does it much faster.

Resolume claim some impressive benchmarks in the use of the codec in Resolume Avenue 3.

Have you found any difference in terms of ATI versus NVIDIA graphics cards?

There is no difference in DXV performance between ATI and NVIDIA. But the faster the GPU, the better the DXV performance is.

How about Windows versus Mac?

Encoding is a bit faster on the PC, but we have not seen much difference in DXV playback performance in Resolume, which is great, because QuickTime video playback is usually slower on the PC. [That's not the case] for DXV, because we bypass a lot of the QuickTime API.

Any chance we’ll see DXV in use beyond Resolume – what about other VJ apps?

Yes, it can be used in other software; The Pixel Addicts and UVA both use it in their high-end media servers. If any other VJ software is interesed in supporting it then we’d love to hear from them.

Ed.: Note – we didn’t get a solid answer as to how this would be licensed or arranged, so you’re on your own, folks!

We did get readers asking about that specifically, wondering if they would want to add an additional encoding step for video that then wouldn’t be usable with other VJ apps. Obviously not an issue if you’re planning a set with Resolume Avenue 3 exclusively.

Indeed if compatibility with other VJ software is required, then DXV is not a good solution yet. But if you’re looking for the best possible performance, then DXV is great.

Since DXV is decoded on the GPU, won’t that have an impact on the performance of GPU-based video effects and shaders?

Using DXV will not slow down any other things rendering on the GPU (like video effects) because it uses features that are otherwise not used. DXV does not use shaders so you can use any shader-based effects without reducing their performance.

Ed.: I tried to get some more specifics on this, but Resolume is holding some of their cards close to their chest as far as how DXV actually works, as their secret herbs and spices! Suffice to say, they claim if you run the benchmarks, you’ll be a believer. Specifically, Bart responded —

“Just saying that it doesn’t use shaders is already [saying a lot.] Benchmarks will do the rest to convince people effects really are not slower with DXV.”

Using a sample from Resolume’s HD-resolution Orange Strings footage pack, you can give the new codec a go on your own rig, without any encoding first.

How well does DXV work when scratching video?

Performance when scratching and jumping in the video is excellent because every frame is a key-frame. DXV uses spatial compression, not temporal compression.

What are you using on the back end? Is this still making significant use of the QuickTime API?
No, it only uses small parts of the QuickTime API.

What machines are you testing on in your rig?

We have a couple of MacBooks Pro and an 8 core Mac Pro with ATI and NVIDIA cards. All our machines run Mac OS X and Windows.

Unibody MacBook Pro 2.4 GHz Intel Core 2 Duo
7200 RPM Hard Drive 2GB RAM
NVIDIA GeForce 9400M + 9600M GT 256MB

MacBook Pro 2.33 GHz Intel Core 2 Duo
7200 RPM Hard Drive 2GB RAM
ATI Radeon X1600 256MB

2x 2.80 GHz Intel Xeon Quad Core
7200 RPM Hard Drive 2GB RAM
ATI Radeon 2600 XT 256MB
Nvidia 8800 GT 512MB

In a world in which 640×480 VJ sets are pretty standard, and 320×240 is a pretty recent memory, multi-layer 1920×1080 is, of course, a pretty big deal. Resolume 3 is still hot off the presses, but can you talk about how these capabilities are being used so far? (And when do we get HD-ready visual clubs? I’ve got a few ideas for a Resolume cocktail to serve at the bar…)

HD mixing is one possibility but also multi-screen performances where you feed two or three screens from a large composition in Avenue. Eyesupply uses the DXV Codec for this with great success on large events with multiple LED screens.

Below: some of the good stuff Eyesupply is up to, via MySpace.

Eyesupply 2008 Reel

Links:
http://www.resolume.com/avenue/dxv.php
http://www.thepixeladdicts.com and their Media Server
http://www.uva.co.uk

  • http://www.sparkav.co.uk toby*spark

    Glad to see they're potentially opening it up – neither love nor money could persuade Dave a year or so back. I've been harping on about a proper vj codec for years: super performance is a bonus, scratchable alpha a necessity imho.

  • Beebles

    Woot woot! Nice interview – I'd *really* like to see how DVX compares to MJPEG (32 and 64) in benchmarks, but without any news on if/how Resolume might licence DVX, I suppose it's a moot point. Still, it you want to see the codec titan's battle, these are the ones! (at least for Windows systems).

    MJPEG:
    http://www.pegasusimaging.com/picvideomjpeg.htm

  • http://vade.info vade

    Thanks, this is certainly interesting. If it isnt shader based then it probably is some sort of compressed texture based format? I dont know.. but I think a trip to OpenGL Profiler is in order :)

  • http://vade.info vade

    Yeah, I just took the liberty to poke around a bit with the dev tools, and it looks like DVX is a QT wrapper for the common DXT1 texture compression format.

    Thats a really smart thing to do!

  • VJ Air

    Beebles….. i ran tests with Picvideo Mjpeg and DVX codec's in Avenue. i encoded the same files at the same framerates ( 15fps ) and the same res ( 1280 x 720 ) 4 layers of DVX outperformed 2 layers of Picvideo Mjpeg… ie, i had a higher framerate with the DVX files.

    im going to do some 1080p tests tonight, but i think i know roughly what the results will be already.

  • Beebles

    ^VJ Air – thanks for sharing your findings!

  • http://vade.info vade

    I ran some more profiling tests on my Mac Book Pro 2.33Ghz with ATI x1600.

    Running with the included sample media with Avenue, and with the downloadable DVX sample media, no effects, 1 video layer, output enabled windowed on main display, and lots of open background apps and no reboot (im too lazy to hook up a second monitor etc)

    playing back DVX to the GPU used a paltry 1 – 4% of GL / GPU timeslice and ~50-60% CPU,

    playing back the included AVI – cinepack decoder in QT used approximately 10-15% of the GL/ GPU timeslice and ~60-70% CPU

    converting that AVI to PhotoJpeg @ high resulted in 8-13% GPU timeslice and ~70% cpu usage

    converting that AVI to AIC resulted in about the same as PhotoJpeg.

    The gl call for DVX im measuring is glCompressedTexSubImage2D, vs glTexSubImage2D for uncompressed image uploading.

    Very very interesting. Need to reverse engineer how they are getting QT to not decompress DVX frames on the CPU. Should be interesting.

  • http://vade.info vade

    Oops, I meant DXV! Sorry, dyslexia :)

  • VJ Air

    doh, there must be something about the name, i keep typing it the wrong way round too!

  • http://www.colour-burst.com cat

    Think I should make a note that the Hippotizer has been using hardware decompression for a while now using I-frame mpg2 and nvidia clearvid effectively doing the same thing as DXV, as has MXwendler with their own format, nice thing here is that it may become a format other developers can use as well, as hippo and mxwendlers decompression are both closed and unlicencable. I'm very pleased to see this released, as I heard about it a while ago!

  • ahoeben

    <q cite="vade">it looks like DVX is a QT wrapper for the common DXT1 texture compression format</q>
    The name of the codec was a bit of a giveaway, and the fact that every frame is a keyframe only confirmed it. Smart indeed.

  • Pingback: DXV codec for Resolume. A new codec for VJ’s?

  • http://www.granulatedsugar.org Caden Alexander

    Tom Cruise have dyslexia and yet he is still a very successful actor.;'*