AudioTouch and more

Well, it’s about time I released another application right? If anyone has ever played Bloom on the iphone, I’m sure you’ll recognize my port from iphone to table.

If you’re not familiar with bloom, it’s a very simple music creation application. Press anywhere on the screen to create circles that grow and ‘bloom.’ As they fade out and re-appear, they play a note. The note is based on the x-axis. As you move right, the notes go higher.

How to use:

-Press on screen (mouse, touch, multi-touch)
-Rollover/touch/click already created circles to remove them and their sound
-Press the upper left ‘clear’ button to clear all circles and sounds on screen.


I’ve posted a browser version here:

Source Code:

Binary Application:

The source folder should include everything you need to run the application. You don’t need to copy anything into your current AS3 directory. I’ve commented the file very well and it should give a good look into making simple applications with the AS3 Touch API.

Thanks to Christian for the background color tweening effect.

Check out my Bloom thread I started here:

Check out olo:idea video below that shows the Bloom running on a large multitouch display.

Check out bassmang5 video below that shows the Bloom and my other applications running on his MTmini! You can see his post on NUIGroup here:

Here’s some pictures from the second event where AudioTouch was on display.

It’s been a loooong way for many of you, but finally the new tracker has been made public!

tbeta promotional screenshot

tbeta promotional screenshot

Checkout the tbeta website where you can download and read the getting started full guide to see how easy it is to use!

note: if you like the old touchlib tracker better, you can still use it if you’re on windows.

Well, I haven’t posted in a while, but that doesn’t mean I’ve stopped development. Actually, I’ve done the opposite and have been developing with a release date coming soon (hopefully).

I applied for GSoC a couple months ago and my project was chosen and I am have been a student under Natural User Interface Group (NUIGroup) ever since. For my project, I’ve been working on a multitouch framework/lux.

For the last few weeks (with the help of my mentor Christian Moore) I’ve been working on porting over touchlib (the main tracker of the NUIGroup community) to openframeworks. A few reasons we chose to use openframeworks is because it’s (1) Cross-platform (it’ll work on windows/mac/linux), (2) It’s easy to add and develop on (minimal coding experience required).

If you didn’t catch that, this means we will have a new, full functional tracker that works on Mac! Many have been waiting for this! =)

So, here’s the summary. The new tracker has the following features:

  • Filters (dynamic background subtraction, highpass, amplify/scaler, threshold) – This means it’ll work with all optical setups (FTIR, DI, LLP, DSI). More filters can be added later (for optimization), but the current filters work for all setups. Also, you no longer need an ‘invert filter.’ Whether you’re working with dark blobs or light blobs, the tracker will work the same!
  • Camera switcher – Have more than one camera on your computer? Now you can press a button and switch to the next camera on your computer without having to exit the application.
  • Input switcher – Want to use test videos instead of a live camera? Go ahead, press a button and it’ll switch to video input.
  • Input flipper – Now you can flip the camera vertical or horizontal if it’s the wrong way.
  • Dynamic mesh calibration – For people with small or large tables, now you can add calibration points (for large displays) or create less points (smaller displays).
  • Warp – Used for getting a rough calibration. By moving 4 points on the source image, you can warp the image to the projection area which will quickly calibrate your setup. This is pretty CPU heavy though, but may work well for people without projection (MTmini) and need to calibrate some.
  • TUIO Send – Although their will be a non GUI mode/version of the application which will have greater speed, you can send OSC TUIO messages directly from the configapp for quick testing (no need to exit and load a seperate file).
  • Camera and application FPS viewer – Now you can see the framerate of both the tracker and camera that you’re getting.
  • Demo view – Why should you have to exit the config tracker to try a quick demo? Now you can test a simple demo and make sure calibration is how you want it.
  • New GUI – The new interface is hopefully more intuitive and easier to understand/use than in the past.
  • Cross-platform – This will work on windows, mac, and linux! Whoo!

Screenshot of new tracker (still in development)

I can’t give out all the deailts, but many more features (fiducial tracking😉 ) are being discussed for further development. Stay tuned for the release and arrival of LUX which will make this whole process of developing multitouch applications a lot more organized! =)

So the questions is…when will this be released? Unfortunately, I don’t give release dates since I don’t like disappointing people if it doesn’t meet the deadline. Fortunately, I can say that we’re not far from making this available relatively soon (i’ll let you guess what that means).

Feel free to comment with your thoughts, hopes, and dreams…. =)

Pictures from when my table was on display for a few days. I have larger pictures if you want specific ones. This post has a lot of pictures, so if they don’t all load, you may need to refresh this page.

New video of the new system that loads and unloads applications. Also, the desktop is now sending out OSC message to PureData to create some sounds based on finger number and position on screen.

In such a short amount of time I’ve hit 1000 downloads! Well, actually it’s 1062 to be exact. Thanks everyone who has been testing and downloading the applications. Feel free to let me know how everything is running. I’d love to hear more feedback.

On another note, the table is about ready for display! It features 4 access points that allow people to load a menu where they can select applications to load. Many applications can be loaded and at the same time. Pressing on the desktop creates wind and glitch type sounds. There’s currently 4 other applications that can be loaded which include: Musical Wong (to be released soon), Musical Squares, Audioshape Sequencer, and MultiKey (half and whole keyboard).


My name is Seth Sandler and I am an undergraduate student finishing a Bachelors degree in Interdisciplinary Computing and the Arts with an emphasis in Music at the University of California, San Diego.

I'm currently doing research and developing a Multi-user, Multi-touch musical interface. This blog space will serve as both a documentation of my own progress as well as an informational space for others to learn and peruse their own Multi-touch techniques and applications.


Want to build your own multitouch surface table? Go to NUI Group to find out how.

Twitter – Find me there!


Aesthetic Goal

Original Concept

Final Result

Want to build your own multitouch surface table? Go to NUI Group to find out how.

AudioTouch Current Setup

Unibrain Fire-i Color camera
Swann Wide-angle IR Camera Lens
50 Osram SFH485 IR LEDs
4 Breadboards
Acrylic 24" x 36"
Wood Frame
Sorta Clear 40 Silicone Rubber
Rosco Grey Screen
Projector Mirror

Blog Stats

  • 400,313 hits

Current Viewers

hit counters