Processing OpenGL Tutorial Video #2- Bubbles! from Andy Best on Vimeo.

Interested in performing high-performance, high-quality video processing, computer vision, motion tracking, and analysis? And want to do it in the friendly Processing coding environment – an ideal place to start, even for non-programmers? First, you’ll want to read Andy Best’s introduction to OpenCV posted a few days ago, to get started with the topic:

Processing Tutorials: Getting Started with Video Processing via OpenCV

But we’ve got next steps for you, as well. Andy has added a second tutorial which begins to cover actual motion analysis. It’s a simple technique, one possible even in Flash – but with OpenCV and Java/Processing, it can run very efficiently, and it’s a good stepping stone to more sophisticated techniques. Andy writes:

In this tutorial, I will show you how to use a thresholded frame difference (motion) image in order to perform collision detection with objects onscreen. Essentially we will be creating something similar to one of the old webcam games where you can ‘pop bubbles’ with your hands (or indeed anything that moves).

Processing OpenCV Tutorial #2- bubbles [andybest.net]

For another trick, here’s reader naus3a playing with OpenCV’s face recognition algorithm. I’ll let you figure out what to do with this one (but it could make an interesting performance tool … hmmm).



Andy also points us to two other powerful techniques using other libraries, as seen on Bryan Chung’s blog:

Augmented Reality Library in Processing
Gesture Recognition in Processing 1

The gesture recognition toolkit uses a mouse as input, but the idea could be applied to video, as well.

Good stuff, and we’ll keep following applications and keep the tutorials coming. Let us know how you fare, and if you get stuck / have any questions.

One little point on Andy’s tutorial. Andy says: “The fact that Processing is based in Java means that while it isn’t as fast as a higher level language like C, it makes up for it with the huge amount of libraries availiable to the Java platform. If speed is what you’re after though, I’d definitely recommend the C++ framework OpenFrameworks.” Now, I’m a big fan of OpenFrameWorks. But in fact, when you’re using OpenCV inside Processing, you should get performance roughly equivalent to OpenCV in OpenFrameWorks/C++, because OpenCV is built with C++. And it is possible to get high performance with Java (which is a higher-level, not lower-level language than C), though the actual variables there can be complex, having to do with the garbage collector, whether you want real-time performance, and what libraries you’re using. Anyway, best to think of these as separate tools, and certainly you should feel perfectly confident in using Java/Processing if you like!

  • http://www.danreetz.com daniel reetz

    These works with OpenCV and Processing are really exciting.

    What are the chances you'd do an "ask the community" post on the state of webcams with Processing on Windows/Mac/Linux? We could all do more vision if we had working cameras…

  • http://www.andybest.net Andy Best

    I guess it is true what you say about all of the OpenCV stuff being just as fast due to the library being in C++, but I find that the performance hit comes when writing the OpenCV image onto the Processing window (stage?). I'm not sure if it is just me, but I'm finding it a lot faster to use a blend() command to put the image into the window than just using image().

    @daniel reetz As I understand it, webcam support is pretty good on Windows (being that most webcams will have windows drivers). I believe it depends on the video capture method that you use though (you might want to try different libraies to see which one works best for you- processing.video, jMyron, OpenCV capture etc). As for OS X, I believe that the webcam usually has to be compatible with QuickTime, however, there is an open source driver that has support for quite a few webcams here http://is.gd/14jH (Support for some cams is better than others).

  • http://createdigitalmusic.com Peter Kirn

    Ah, yes — now that's because PImage and image() specifically weren't really intended for high performance or video. You can use GSVideo and GLGraphics for a much more efficient solution there.

    I'd love an ask the community on working webcams; sounds like a great idea. Was there something specific you wanted to ask, or just to see what folks are using?

  • http://www.andybest.net Andy Best

    Peter Kirn- I shall have a look into them, thanks for the tip :)

  • naus3a

    i really want to get my hands wet with GSVideo: my computer vision sketches work quite smooth, but when i add a mixed reality layers stuff begin to get heavy. by the way i'm working these days on a real world mixed reality presentation/gig tool… it could be a nice test

  • http://www.danreetz.com daniel reetz

    Andy, you're correct that most webcams have a driver on Windows but my previous experience with Processing and WinVDIG was totally miserable. I tried three different webcams, and ended up only getting one of them to work, at a miserable 320×240. I'll check out the different libraries, thanks.

    I could try to outline a question, but maybe outlining a goal makes more sense — what would be cool is if we'd all find that one model of webcam that had good enough performance (say, VGA at 30fps) and worked on and with *everything*, ideally with a "native" driver — the community model I've seen before is the way the wireless hacking community finds well-performing, permissive wireless cards that work on Windows, Linux, and Mac. Macam, linked above by Andy, looks pretty close.

    I mean, sure, I can buy the Logitech whatever, install their sheds of crapware, and it will *probably* work (or I can return it and lose only the hours of screwing around with it), but it's hard to find these experiences and this kind of information in a database or forum. Perhaps that's because for the people it works for, it works well and without much effort, and for the others, it's too frustrating to get started. Having spent hours playing with different cameras and Googling around, the input of people with working setups would be invaluable.

  • http://createdigitalmusic.com Peter Kirn

    Hey Daniel,
    WinVDIG is a disaster. Avoid at all costs. In fact, I think it's fair to say with Processing NO ONE should be using the default QuickTime library unless they're on a Mac and really desperate. The path forward I believe is going to be GSVideo wrapping native support on each platform (even QuickTime Java, on which Processing's default video library is based, doesn't really count as "native" on the Mac, because it's a deprecated, dead library).

    GSVideo or the native capture routines in OpenCV will both serve you well.

    Oh, and also avoid Logitech at all costs. (I know, I'm full of great news here.) I've seen major conflicts between their drivers and audio, and other significant sources of instability. Certainly, you don't want to install their video driver.

    But I agree absolutely — shared video knowledge would be huge. Maybe we can even have a little Processing install / test party with these libraries and cameras on IRC for cdmotion. ;)

  • http://www.danreetz.com daniel reetz

    Certainly, you don’t want to install their video driver.

    Yeah, pretty sure that's what hosed me in the end — the multiple driver installs irreversibly destroyed my ability to get new cams working. So, OK, no Logitech (no surprise, and I've had similar experiences with Creative products) and no WinVDIG (from your advice and my experience)… Which webcam to use, then?

  • http://www.danreetz.com daniel reetz

    And sure, I'd install an IRC client for such a party. ;)

  • http://createdigitalmusic.com Peter Kirn

    Well, any driverless camera is your best first bet. I think Creative and even Logitech have a class-compliant camera… mine is a Live! Cam Optia or something? I'll write it up.

  • Pingback: Bruno Campagnolo de Paula weblog » Resumo do dia para 2009-02-11

  • Pingback: R.Seiji » links for 2009-02-24

  • JasononCDM

    I've installed the Open CV folder in libraries and now I'm getting an "unexpected token:|"error.

    The line highlighted is: Go to Previous message | Go to Next message | Back to Messages

  • adambentley

    thanks for the tutorials. does anyone have experience using integrated laptop cameras with opencv and processing?

  • http://google Suhas

    i am having problem to run the OPen cv program plz help me my program is

    import java.awt.*;
    import java.awt.event.*;
    import java.awt.image.MemoryImageSource;
    import hypermedia.video.OpenCV;

    public class FaceDetection extends Frame implements Runnable {

    // program execution frame rate (millisecond)
    final int FRAME_RATE = 1000/30;

    OpenCV cv = null; // OpenCV Object
    Thread t = null; // the sample thread

    // the input video stream image
    Image frame = null;
    // list of all face detected area
    Rectangle[] squares = new Rectangle[0];

    /**
    * Setup Frame and Object(s).
    */
    FaceDetection() {

    super( "Face Detection Sample" );

    // OpenCV setup
    cv = new OpenCV();

    cv.loadImage("c://17.jpg",200,200);
    // cv.capture( 320, 240 );
    cv.cascade( OpenCV.CASCADE_FRONTALFACE_ALT );

    // frame setup
    this.setBounds( 100, 100, cv.width, cv.height );
    this.setBackground( Color.BLACK );
    this.setVisible( true );
    this.addKeyListener(
    new KeyAdapter() {
    public void keyReleased( KeyEvent e ) {
    if ( e.getKeyCode()==KeyEvent.VK_ESCAPE ) { // ESC : release OpenCV resources
    cv.dispose();
    System.exit(0);
    }
    }
    }
    );

    // start running program
    t = new Thread( this );
    t.start();
    }

    /**
    * Draw video frame and each detected faces area.
    */
    public void paint( Graphics g ) {

    // draw image
    g.drawImage( frame, 0, 0, null );

    // draw squares
    g.setColor( Color.RED );
    for( Rectangle rect : squares )
    g.drawRect( rect.x, rect.y, rect.width, rect.height );
    }

    /**
    * Execute this sample.
    */
    public void run() {
    while( t!=null && cv!=null ) {
    try {
    t.sleep( FRAME_RATE );

    // grab image from video stream
    cv.read();

    // create a new image from cv pixels data
    MemoryImageSource mis = new MemoryImageSource( cv.width, cv.height, cv.pixels(), 0, cv.width );
    frame = createImage( mis );

    // detect faces
    squares = cv.detect( 1.2f, 2, OpenCV.HAAR_DO_CANNY_PRUNING, 20, 20 );

    // of course, repaint
    repaint();
    }
    catch( InterruptedException e ) {;}
    }
    }

    /**
    * Main method.
    * @param String[] a list of user's arguments passed from the console to this program
    */
    public static void main( String[] args ) {
    System.out.println( "
    OpenCV face detection sample
    " );
    new FaceDetection();
    }

    }

    it gives error like this ::
    !!! required library not found : C:Program FilesJavajdk1.6.0_10jreinOpenCV.dll: This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem
    Verify that the java.library.path property is correctly set and the 'path oOpenCVin' exists in your system PATH

    Exception in thread "main" java.lang.UnsatisfiedLinkError: hypermedia.video.OpenCV.loadImage(Ljava/lang/String;II)V
    at hypermedia.video.OpenCV.loadImage(Native Method)
    at face12.FaceDetection.(FaceDetection.java:55)
    at face12.FaceDetection.main(FaceDetection.java:132)
    Java Result: 1

  • Pingback: Create Digital Motion » Set Up Ubuntu Linux for OpenCV

  • Pingback: Create Digital Motion » Computer Vision Goodness: OpenCV 2.0, OpenCV on Android, Book

  • NeagPealPat

    Hi guys.
    My PC worked slowly, too much errors. Please, help me to fix errors on my computer.
    I used Windows XP.
    Thanks,
    NeagPealPat

  • sindhu

    Exception in thread "main" java.lang.UnsatisfiedLinkError: hypermedia.video.OpenCV.loadImage(Ljava/lang/String;II)V

    at hypermedia.video.OpenCV.loadImage(Native Method)

    at FaceDetection1.(FaceDetection1.java:68)

    at FaceDetection1.main(FaceDetection1.java:310) can anyone tell me wats this i ahve included all d jars&dll and also i have set the system path to opencv bin ,,, help me in rectifying this at the earliest

    thx in advance