OpenCV Motion Tracking, Face Recognition with Processing: I’m Forever Popping Bubbles
Processing OpenGL Tutorial Video #2- Bubbles! from Andy Best on Vimeo.
Interested in performing high-performance, high-quality video processing, computer vision, motion tracking, and analysis? And want to do it in the friendly Processing coding environment – an ideal place to start, even for non-programmers? First, you’ll want to read Andy Best’s introduction to OpenCV posted a few days ago, to get started with the topic:
Processing Tutorials: Getting Started with Video Processing via OpenCV
But we’ve got next steps for you, as well. Andy has added a second tutorial which begins to cover actual motion analysis. It’s a simple technique, one possible even in Flash – but with OpenCV and Java/Processing, it can run very efficiently, and it’s a good stepping stone to more sophisticated techniques. Andy writes:
In this tutorial, I will show you how to use a thresholded frame difference (motion) image in order to perform collision detection with objects onscreen. Essentially we will be creating something similar to one of the old webcam games where you can ‘pop bubbles’ with your hands (or indeed anything that moves).
Processing OpenCV Tutorial #2- bubbles [andybest.net]
For another trick, here’s reader naus3a playing with OpenCV’s face recognition algorithm. I’ll let you figure out what to do with this one (but it could make an interesting performance tool … hmmm).
Andy also points us to two other powerful techniques using other libraries, as seen on Bryan Chung’s blog:
Augmented Reality Library in Processing
Gesture Recognition in Processing 1
The gesture recognition toolkit uses a mouse as input, but the idea could be applied to video, as well.
Good stuff, and we’ll keep following applications and keep the tutorials coming. Let us know how you fare, and if you get stuck / have any questions.
One little point on Andy’s tutorial. Andy says: “The fact that Processing is based in Java means that while it isn’t as fast as a higher level language like C, it makes up for it with the huge amount of libraries availiable to the Java platform. If speed is what you’re after though, I’d definitely recommend the C++ framework OpenFrameworks.” Now, I’m a big fan of OpenFrameWorks. But in fact, when you’re using OpenCV inside Processing, you should get performance roughly equivalent to OpenCV in OpenFrameWorks/C++, because OpenCV is built with C++. And it is possible to get high performance with Java (which is a higher-level, not lower-level language than C), though the actual variables there can be complex, having to do with the garbage collector, whether you want real-time performance, and what libraries you’re using. Anyway, best to think of these as separate tools, and certainly you should feel perfectly confident in using Java/Processing if you like!







17 Comments
Leave a Commentdaniel reetz
These works with OpenCV and Processing are really exciting.
What are the chances you’d do an “ask the community” post on the state of webcams with Processing on Windows/Mac/Linux? We could all do more vision if we had working cameras…
February 11th, 2009 @ 7:19 am
Andy Best
I guess it is true what you say about all of the OpenCV stuff being just as fast due to the library being in C++, but I find that the performance hit comes when writing the OpenCV image onto the Processing window (stage?). I’m not sure if it is just me, but I’m finding it a lot faster to use a blend() command to put the image into the window than just using image().
@daniel reetz As I understand it, webcam support is pretty good on Windows (being that most webcams will have windows drivers). I believe it depends on the video capture method that you use though (you might want to try different libraies to see which one works best for you- processing.video, jMyron, OpenCV capture etc). As for OS X, I believe that the webcam usually has to be compatible with QuickTime, however, there is an open source driver that has support for quite a few webcams here http://is.gd/14jH (Support for some cams is better than others).
February 11th, 2009 @ 8:28 am
Peter Kirn
Ah, yes — now that’s because PImage and image() specifically weren’t really intended for high performance or video. You can use GSVideo and GLGraphics for a much more efficient solution there.
I’d love an ask the community on working webcams; sounds like a great idea. Was there something specific you wanted to ask, or just to see what folks are using?
February 11th, 2009 @ 8:42 am
Andy Best
Peter Kirn- I shall have a look into them, thanks for the tip
February 11th, 2009 @ 8:53 am
naus3a
i really want to get my hands wet with GSVideo: my computer vision sketches work quite smooth, but when i add a mixed reality layers stuff begin to get heavy. by the way i’m working these days on a real world mixed reality presentation/gig tool… it could be a nice test
February 11th, 2009 @ 11:16 am
daniel reetz
Andy, you’re correct that most webcams have a driver on Windows but my previous experience with Processing and WinVDIG was totally miserable. I tried three different webcams, and ended up only getting one of them to work, at a miserable 320×240. I’ll check out the different libraries, thanks.
I could try to outline a question, but maybe outlining a goal makes more sense — what would be cool is if we’d all find that one model of webcam that had good enough performance (say, VGA at 30fps) and worked on and with *everything*, ideally with a “native” driver — the community model I’ve seen before is the way the wireless hacking community finds well-performing, permissive wireless cards that work on Windows, Linux, and Mac. Macam, linked above by Andy, looks pretty close.
I mean, sure, I can buy the Logitech whatever, install their sheds of crapware, and it will *probably* work (or I can return it and lose only the hours of screwing around with it), but it’s hard to find these experiences and this kind of information in a database or forum. Perhaps that’s because for the people it works for, it works well and without much effort, and for the others, it’s too frustrating to get started. Having spent hours playing with different cameras and Googling around, the input of people with working setups would be invaluable.
February 11th, 2009 @ 1:03 pm
Peter Kirn
Hey Daniel,
WinVDIG is a disaster. Avoid at all costs. In fact, I think it’s fair to say with Processing NO ONE should be using the default QuickTime library unless they’re on a Mac and really desperate. The path forward I believe is going to be GSVideo wrapping native support on each platform (even QuickTime Java, on which Processing’s default video library is based, doesn’t really count as “native” on the Mac, because it’s a deprecated, dead library).
GSVideo or the native capture routines in OpenCV will both serve you well.
Oh, and also avoid Logitech at all costs. (I know, I’m full of great news here.) I’ve seen major conflicts between their drivers and audio, and other significant sources of instability. Certainly, you don’t want to install their video driver.
But I agree absolutely — shared video knowledge would be huge. Maybe we can even have a little Processing install / test party with these libraries and cameras on IRC for cdmotion.
February 11th, 2009 @ 1:40 pm
daniel reetz
Certainly, you don’t want to install their video driver.
Yeah, pretty sure that’s what hosed me in the end — the multiple driver installs irreversibly destroyed my ability to get new cams working. So, OK, no Logitech (no surprise, and I’ve had similar experiences with Creative products) and no WinVDIG (from your advice and my experience)… Which webcam to use, then?
February 11th, 2009 @ 1:48 pm
daniel reetz
And sure, I’d install an IRC client for such a party.
February 11th, 2009 @ 1:49 pm
Peter Kirn
Well, any driverless camera is your best first bet. I think Creative and even Logitech have a class-compliant camera… mine is a Live! Cam Optia or something? I’ll write it up.
February 11th, 2009 @ 1:50 pm
JasononCDM
I’ve installed the Open CV folder in libraries and now I’m getting an “unexpected token:|”error.
The line highlighted is: Go to Previous message | Go to Next message | Back to Messages
March 4th, 2009 @ 4:54 pm
adambentley
thanks for the tutorials. does anyone have experience using integrated laptop cameras with opencv and processing?
March 27th, 2009 @ 12:05 am
SysGOD
http://augmented.taba.ru/ My page with video of Ogre3D augmented reality
May 30th, 2010 @ 12:17 pm
Shervin Emami
If you want to learn how to do Face Detection and Face Recognition in OpenCV, there is a tutorial and source-code at
http://www.shervinemami.co.cc/faceRecognition.html
It also has some other OpenCV tutorials that you might like.
June 8th, 2010 @ 5:02 pm
Leave a comment
RSS feed for comments on this post. TrackBack URI