beach

It’s a beach.longerbeach
Now, it’s a more longer beach. Hmmm… too bad you can’t do this to the real world.

“Content-aware” image resizing — the ability to stretch images without distortion — is all the rage. vade covered the technology at last summer’s SIGGRAPH, and we’ve since seen publicly-available tools. But the New York Times musters an entire feature story on the topic (now it’s definitely mainstream), complete with a monster round-up of tools.

Thanks to Emmet for the tip!

Stretching the Truth Just Became Easier (and Cheaper) [NY Times; free registration required]

And the tools:

Liquid Rescale, free for use online or as a GIMP plug-in

Liquid Resize, free standalone Windows app

onOne Software, apparently commercial product forthcoming

RSizer, free Flash tool for resizing

IntuImage, Windows-based image editor

And, while you’re at it:

VectorMagic, free Web vectorization tool

Microsoft GroupShot, for merging images (and incidentally, there are some surprisingly hot image-merging tools in Windows Live Gallery for XP/Vista — really. Occasionally, though not nearly often enough, Microsoft notices how insanely cool their R&D people are.)

Okay, so there are a lot of free tools out there for doing cool stuff. The Times suggests this might pose a threat to the hegemony of Photoshop, except that I think Adobe’s sales are fairly safe with all the people who need gorilla-sized programs for image editing — and consumers have long had lightweight alternatives, even since the earlier days of Adobe.

And I did have good fun stretching my snapshot of the Santa Barbara shoreline above.

But here’s what’s missing from this picture: motion. And being that that’s the mission of this site, I think this will all get really interesting when we see two things:

Real-time processing: Finding the stitch/seam points is processor-intensive (well, maybe not as slow as it is in Flash in RSizer, but slow nonetheless). But you can pre-process still images. That means it should be absolutely possible to manipulate images using this technique for live motion. My beach wouldn’t make a terribly great visualist performance, but I’m sure you could find something fun.

Video: vade suggested when we first saw this that this could revolutionize changes of aspect ratio. And lest you think that sounds too processor-intensive, look at the new motion analysis features of tools like Apple’s Motion, which churn through video with growing speed. Finding these seams would appear to take a lot of pixel crunching, so this could be a ways off. But I don’t think it’s necessarily impossible — especially if an algorithm could be developed simplifying the technique for this purpose.

Anyone with data algorithm chops, though, let me know if I’m nuts.

  • DaNni

    I think the transition of this technique to moving images won't be as trivial as you suspect. If you saw the original SIGGRAPH presentation, you'll remember that the resizing works by finding the horizontal and vertical lines and deleting just those. The problem with a sequence of images is that -by its own changing nature- the places where these deletions occur might vary from frame to frame. This effect would cause some kind of wobble wich I suspect would be nauseating (but maybe also funny and interesting… maybe even useful for some VJ sets! :) )
    You could try to implement an algorithm that took this into account and analyzed several frames and made a time-weighted decision (using optical flow or something) but it'd have to be quite complex and nevertheless not successful with every kind of video input (depending on the amount of detail, motion, etc).
    Still, I'm eager to d/l and try some of these tools and play with them.

    PS: Don't worry about Photoshop, I read Adobe hired the guy who invented the technique just a week after the presentation…

  • ian

    when you consider moore's law, how long until a process like this becomes fast as shit, on the average octo-core 6ghz processor with 16 gigs of ram? hold on while i strap into my jetpack so i can get to the digi-office, i'm late for eWork!

  • http://createdigitalmusic.com Peter Kirn

    Actually, I am with DaNni, believe it or not — Moore's Law or not, processing is about orders of magnitude. So what I'm imagining is that some sort of simplified algorithm could be devised. Maybe you even manually mark areas you want to maintain in aspect ratio changes, for instance, to cut down the amount of processing. And I would see it being something you might try to weight based on optical flow or something similar. Not necessarily something close here, but I could see this happening in the not-too-far future.

  • http://www.intuimage.com Serge

    Peter, IntuImage is not Flash-based, it's a native Windows application.

  • http://createdigitalmusic.com Peter Kirn

    @Serge: thanks. Doh. Got my wires crossed going between different link items. Windows options actually aren't too shabby for image editing, I'd like to observe, despite the Mac's better reputation for image tools. Also love paint.net.

  • Pingback: Create Digital Motion » Live Mobile Video Stitching, Omni Projector: Microsoft Research Goodies

  • http://www.umiacs.umd.edu/~jsp Jai Pillai

    I have recently read a paper on Seam carving, which presented very attractive results on context aware image resizing.