Apple Magic Trackpad as Multi-Touch Input, and Cross-Platform Multi-Touch
Apple today, alongside beefed-up iMacs with quad-core and Mac Pro towers with twelve, introduced a $69 Bluetooth trackpad accessory with multi-touch gesture support.
http://www.apple.com/magictrackpad/
I’m not a huge fan of trackpads over mice, but yes, this does give you a cheap multi-touch input to play with. And we haven’t seen much in the dirt-cheap, under-$100 space. Wacom’s Bamboo Pen & Touch is a fun, decent-enough tablet available at a street price about the same as the Magic Trackpad. But its “multi-touch” functionality consists of a set of hard-wired gestures – developers can’t get at discrete touch points directly on the device, even though that functionality is now part of the platform on Windows 7. (Cheap multi-touch sensing often isn’t terribly accurate, so you might not want that data anyway.)
Apple’s Magic Trackpad is different. We’ve already seen great, precise touch input from MacBook trackpads, and now that functionality is available even if you have an older model. (I’m now tempted to pick up one for my black MacBook.)
Gestures are nifty enough – Apple’s Pro Apps support them, so you can use these in Logic or Final Cut, if you like. But what I think creative visualists and musicians will really want is discrete input points. I haven’t had first-hand experience supporting the trackpads, nor can I confirm what’s available on this, but to get the conversation started, here are some resources; I think in our community, we can solve this relatively quickly.
Kineme MultitouchPatch from George Toledo on Vimeo.
The best Mac-based solution I’ve seen yet is the MultitouchPack from Kineme, which we’ve seen with the internal trackpad as well as Apple’s existing Magic Mouse:
http://kineme.net/release/MultitouchPatch/1.0
A complete set of tools using TUIO, the cross-platform, cross-device, open standard for multi-touch control based on the ReacTable platform:
http://www.tuio.org/?software
Cocoa multi-touch framework:
http://hci.rwth-aachen.de/multitouch
Tongseng on GitHub is already a wrapper around the native Mac functionality on trackpads, which I would think might work with the mice and external Magic Trackpads, too:
http://github.com/fajran/tongseng
xTUIO is a Mac-only tool that brings together touch on the platform
http://www.xtuio.com/
fingerpinger is a “hacked” Max external; GPL’ed so if someone wants to build it for Pd, it could be tested there, too.
http://www.anyma.ch/2009/research/multitouch-external-for-maxmsp/
Apparently works quite well; thanks, @oivindi!
(Ideally, though, something less hacky and built around the native APIs, then translated to something higher-level and cross-platform and portable would be the way forward, I think, though this is a good place to start.)
By the way, there’s also a neat utility called jitouch designed for customizing those trackpad gestures, which I imagine may also work with this.
Mac API reference
Two Mac classes are most relevant to the task at hand:
NSTouch
NSEvent
Thanks to Anton for the tip here.
These appear to include multiple pointer support in the NSEvent message, so tracking multiple fingers should be very possible. But I’m curious to hear from some developers about what will and won’t actually work. (One thing I’m not clear on, as a non-Cocoa dev, was how it even maps multiple touch events… unique ID? Are they also mapped to buttons?)
Going cross-platform?
To me, the appeal of using multi-touch really comes alive when you can go cross-platform. That means representing the events themselves in a consistent way (as TUIO does), but also having some frameworks that can work across platforms. In addition to the Mac support (NSEvent/NSTouch) above, as well as what’s available on mobile devices, there’s also native, public touch support in Windows 7, as well as new, finally-standard support in X and Linux.
A great example of how this can look is the extensive MT4j (Multitouch for Java) framework, which presently supports Windows XP/Vista/7 and Linux (tested on Ubuntu). It can easily swap input devices, so you could go from a multitouch table with objects to a trackpad like Apple’s without batting an eye. It’s missing Mac support, however.
With the Magic Trackpad, I can see two ways of going. It’s a Bluetooth device, so hacking access from other platforms aside from Mac OS may be possible. Moreover, though, it’d be great to see some Mac developers step up and start testing and helping develop MT4j on their platform.
If anyone is interested, I’m certainly ready to help support and promote your work here on CDM.
Of course, the first step before wrapping any Java support or something fancy like that is simply to get touch input working on the Mac. If you pick up one of Apple’s shiny, silver new toys, do let us know.
Previously on CDMo:
Multitouch Buffet: Quartz Composer for Touchpads, Magic Mouse; MT4j and PyMT For Everything
Image Courtesy of Apple.
Updated: Macworld gets a hands-on first look. It nicely covers the end user experience, but not some of our more advanced questions.
13 Comments
Leave a Commentdovemouse
this is a cool max external for the mac trackpads http://www.anyma.ch/2009/research/multitouch-external-for-maxmsp/
i’m guessing it would work for this new product…
it’d be great if you could “hijack” it. so it stops functioning as a mouse and the data goes straight to the app you’re using!
July 27th, 2010 @ 5:24 pm
Lee
Ordered it. Should arrive in two days. Kineme Multitouch patches are already installed and in use, so I’m good to go. Should be sweet. Regarding getting it dedicated to an app; for specialized multitouch exclusivity on the Magic Mouse (such as Kineme Multitouch), one simply turns off the unwanted functionalities in System Preferences. I’m guessing the Magic Trackpad would be similar.
July 27th, 2010 @ 5:46 pm
Jordan
Hey Peter,
IMHO opinion, while I don’t think this is the be all end all (nor would I expect it to be), I am really intrigued, particularly for its use in live performance (both audio and visual). There was an interesting NIME talk this year discussing research into gesture/spatial relationships using multipoint devices, e.g. Apple laptop trackpads– creative output can really get interesting when you take into account other spatial relationships between touches (besides the common XY relationships + number of touches…). I can see that work greatly benefiting from somthing like the Magic Trackpad. Excited!
Heres a link to the paper: http://www.educ.dab.uts.edu.au/nime/PROCEEDINGS/papers/Paper%20F1-F5/P136_Schlei.pdf
July 27th, 2010 @ 11:10 pm
Lee
oh oh … Macworld is reporting that current versions of most third-party utilities for enhancing Apple’s laptop trackpads and the Magic Mouse, including Jitouch, aren’t currently working with the Magic Trackpad. Looks like we may be needing updates for those enhancers. I’ll try to get some info from Kineme about the viability of their multitouch patch for Quartz Composer.
July 28th, 2010 @ 2:00 am
Peter Kirn
That’s not surprising… any number of things I imagine could make those go wrong. That’s also why you need a standard API, not hacks. It does appear to me that the NSEvent level, though, is a standard API; I’m just not clear exactly. So do let us know.
July 28th, 2010 @ 2:02 am
Brian
@dovemouse
I agree and also I love that external…and used it in a max4live patch i made very handy.
I love this, the sole reason i use my mbp more than my desktop is cause of the trackpad….now i can finally get one for that
July 28th, 2010 @ 2:05 pm
Christian Frisson
I ported/forked anyma’s [fingerpinger] to Pd using flext. It is available here, source included:
http://www.numediart.org/download/
Part of a toolbox for reusing off-the-shelf input devices featured in (another) NIME 2010 paper:
http://www.educ.dab.uts.edu.au/nime/PROCEEDINGS/papers/Demo%20Q1-Q15/P473_Frisson.pdf
Alternative: Hans-Christopher Steiner’s [multitouch] native Pd object (part of pd-extended).
July 28th, 2010 @ 6:12 pm
gabe
I have never been ever to figure out how to turn off the four-finger gestures on my trackpad. I’ve been using the kineme multitouch patch for a while, but can’t figure this out.
anyone know how to turn these off? for example, use four fingers and swipe down and it reveals all your document windows, and swipe up and it reveals the desktop. nice feature, until you want to use the multitouch patch or something along those lines.
July 30th, 2010 @ 5:54 pm
Lee
@ Peter Kirn: The Kineme MultitouchPatch seems to be working on the Magic Trackpad.
@gabe: I’m able to turn off four finger Exposé activation in the Trackpad preference panel located in System Preferences. The Trackpad preference panel doesn’t show up there until doing a Software Update and installing the trackpad software (after pairing the Magic Trackpad with the computer), however. If you’ve done all that and it’s still not working … I haven’t a clue what the issue is.
July 31st, 2010 @ 4:20 am
usefuldesign.au
@dovemouse: “it’d be great if you could “hijack” it. so it stops functioning as a mouse and the data goes straight to the app you’re using!”
That would be a super hack! If somebody comes up with it be sure to post back at kineme.net where I read all threads!
I’m thinking along the lines of an overlay with multiple sliders and on/off switches for a room lighting & a/v control device. For $69 that’s a lot cheaper than a set of say 10 midi sliders/knobs/toggles and you can set it up just how you like. The overlay would presumably need the touchable sections cutout of a rigid plastic layout sheet or something.
Congrats on publication gtoledo!
August 1st, 2010 @ 3:55 am
George Toledo
Consider two scenarios:
1: An Interactive Graphics Engine cobbles together a poorly written library to try to replicate some kind of functionality that is in the Mac.
2: An Interactive Graphics Engine is modified, using techniques that Apple uses, to access frameworks that are used in the OS or in major Apple apps, that are unlikely to change until an actual new OS release.
Which one is more of a hack?
Using a private framework that’s for something very app-specific and constantly updated, a blazing new tech that has very minimal use, or a framework that’s for a Pro app may be really_bad_ideas, but for an API that’s already grounded by tons of devices in the field, not so much. As was seen, there were no problems with the magic trackpad, and there was a giant collective yawn
Meanwhile, things like jittouch DIDN’T work.
Which is more of a hack?
August 8th, 2010 @ 4:28 pm
Peter Kirn
@George – not sure anyone’s going to argue with that.
That said, I think unrelated to what jittouch did, I remain interested in whether for cross-platform app development we can talk directly to the device. That appears to be how the magic mouse has already been used on Linux, via a kernel module that directly supports the chip. (That’s *also* not a hack… far from it, then you support the hardware as long as the chip is in there, no Apple frameworks needed.)
August 8th, 2010 @ 6:00 pm
Chris
In the latest trunk on the MT4j google code svn there is an input source which supports the multi-touch mac trackpads. It will probably also work with the magic trackpad which I can hopefully test soon.
August 11th, 2010 @ 12:17 pm
Leave a comment
RSS feed for comments on this post. TrackBack URI