OpenCV powers a multitouch interface, built in openFrameworks. Project by / photo (CC) Todd Vanderlin.

When Thanksgiving rolls around, OpenCV will be one of the gifts for which I’ll be thankful. OpenCV is a “computer vision” library, capable of tracking motion and analyzing images, but generally useful as a pixel-crunching video library for many tasks. It’s a native, C/C++ library, accessible from many other languages, including Java and Python. Experienced C++ programmers and artists dabbling in Processing alike can use it. (See our Processing tutorial.) It’s also a real success story for open source code, first developed by Intel but heavily developed by researchers and artists ever since. And it’s a success story for the permissive BSD License, used in everything from academic work to commercial projects.

I’m happy to have a few bits of really good news on the OpenCV front.

OpenCV 2.0: Vision improved. OpenCV itself has taken a massive leap forward this fall with version 2.0. The library is smarter, faster, leaner, and easier to use. You can program with Python instead of C++. And even if your programming skills are moderate, this should mean lots of new features that will filter into other OpenCV-based libraries. Among the 2.0 features already rolling out in pre-release versions:

  • Loads of new object detectors and descriptors, including a 3D feature descriptor, self-similarity, new abilities to detect objects and people
  • Faster and smater algorithms, and more samples to learn how to use them
  • A new Python interface
  • All-new, cleaner C++ interface
  • Easier installation, easier builds on different platforms, and easier integration with IDEs like Xcode, Eclipse, and CodeBlocks
  • Subversion for access to code
  • Stereo camera support on Linux

Check out the changelog. And to get started, learn how to use the thing, and download OpenCV, see the wiki:

A video application for Android (this one’s actually doing live streaming, but you get the idea). Photo: tomsun.

Computer vision on Android: The other news is that, thanks to Google’s growing support for native code (finally), OpenCV is coming to Android. With OpenCV doing the geeky pixel crunching behind the scenes, you’re free to write some fairly straightforward Java to make vision-enabled mobile apps – or simply use OpenCV for high-performance pixel processing.

You can check out the project on GitHub:

This is an early build, so expect some wrinkles. But there are already some handy tools for getting started. There’s a video emulator, so if you don’t yet own an Android because you’re waiting for the better devices coming out in the next few months (smart), you can still get going. And there’s even some basic sample code for testing with face tracking.

If I’ve fired up your appetite, and you’re not sure where to begin, there is an excellent book on the topic, called – aptly enough – Learning OpenCV. It came out last fall, and so doesn’t include all the 2.0 features, but the basic functionality is the same.

So, would anyone like to start a little user group to work on this stuff? Say so in comments, leave your real email (it shows up for CDM admins, but not publicly), and we’ll get something rolling.

Tutorials from CDM on OpenCV:

Processing Tutorials: Getting Started with Video Processing via OpenCV
OpenCV Motion Tracking, Face Recognition with Processing: I’m Forever Popping Bubbles

…and because Linux is a terrific development platform you can boot into on any Mac, PC, or netbook, check out my rundown of getting OpenCV working on Ubuntu.

  • Kevin Hackett

    This sounds really interesting, but I am a graphic designer, not a coder. How does someone like me who is a visual artist begin the daunting task of learning to use this stuff?. I always dig into it, download a bunch of stuff and then I'm lost. I usually put it all away and wait for the next interesting story to spark my interest in code again. I wanted to make a simple Augmented reality card but there seems to be no "Start Here first instructions." I'm lost in all this code stuff but I love reading about it.

  • Fernando Hashiba

    Really cool, although my recent career involves cinematography, I have a coding background and I've been developing a curricula for a year in interactivity design. I would like to help. Have Fun!

  • RoyMacdonald

    2.0 sounds very nice.
    I've been playing for a while with opencv in Processing and lately in XCode using OpenFrameworks. A week ago I started Reading the book mentioned. I'm really interested in computervision since I feel that there are an incredible amount of things that can be done whit ideas that go from art instalations to comercial/profitable stuff.
    I'd be really glad to be part of a user group in this subject as you, Peter, suggest.

  • Peter Kirn

    @Kevin: see the tutorials above; that's the best way to get started. And Processing I think is the most painless environment for experimentation. (At the same time, there's no reason you can't *stick* to Processing as the projects mature, and even openFrameworks is easier to learn once you've spent some time with the Processing syntax, as OF is really inspired by Processing.)

    As for augmented reality, I'm putting together just that… and that also works on Android. :)

  • Adam

    Count me in for the user group

  • Bill McCord

    I just added some fairly well performing face tracking. Any support on improving the performance of other parts of the system are very much welcome.

  • Pingback: Leave No Trace » I can see the future, and it’s a place about 70 miles east of here

  • Drake Gul

    By the way, if anyone is interested in the far more "hackable" Nokia N900, OpenCV 2.0 compiles out-of-the-box there, without all the mucking about and limitations of an Android port. You can use it with v4l and gstreamer just like on any linux box (as described in e.g. the link "getting OpenCV working on Ubuntu" from the article above.) It really is quite mind-blowingly awesome…

  • shani

    i would love to have some more android-opencv samples using the phone (not the emulator). i am having massive troubles getting it to work. i get it all to compile but keep getting "unsatisfied link errors" and "not finding implementations" of native opencv code even though i've followed all the instructions i've read.

  • Stan Smith

    I see these great apps incorporating video and I wonder about battery use. If GPS sucks the battery dry, does video do that too?

  • Marc Tan

    I'm an Android dev and a bit newbie in image processing. I would love to help in porting opencv functions to Android. Count me in.

  • Tom Reader

    Count me in, too. I'm new to Android and OpenCV, but not to computer vision and programming in general. So two out of four steep learning curves isn't bad… My work is based around barcode images, although I can think of 100 vision tasks to play with that might be *even more* fun than barcodes(!). Looking forward to learning OpenCV, I'll move towards Android after that.

  • Prerana

    Count me too:)…I have implemented Eigenface algo for face recognition using just opencv2.1… i want the same thing on android..Is it feasible??where should i get started with???Since we have opencv ported on android does it provide java wrapper classes for eigenface algo also which i can use in my face recognition??Advices n suggestion are most welcomed…It will be of great help….