Download the program here (7mb)
Download the source code here (GPLv2)
Empty markers can be found here
The research paper can be found here

When doing some research on tangible interfaces, I stumbled upon the concept of augmented reality. Augmented or Mixed Reality is where virtual and real objects appear together in a real time system.. Specifically the marker-based techniques that were used in this type of experimental research were of interest to me. In general, these techniques require a user to first create a digital image and print it; with a webcam and some AR techniques these images could be located and identified with a high degree of certainty, and a position and orientation of the image relative to the camera could be determined. Usually this is then coupled to the transformation of a virtual model, which when superimposed upon the video stream would then appear to stick to the printed marker.

The ARToolKit and ARTag projects made use of markers that look like a black square with some symbol in it. The algorithms used first scan the video stream for black squares and then compare the contents of the squares with known markers. I felt there was an opportunity here to use a modified approach – instead of comparing the contents in order to identify a marker it should be possible to interpret it as a virtual model.  The idea is simple – draw on paper and view it as a computer model on your screen. No mouse/keyboard required. The computer model may augment the sketch in various ways – for example making it 3D, identifying paths, recognizing symbols etc. As a proof of concept I created the ARX program, which assumes you’re trying to draw a simple racetrack. It was implemented pretty quickly, and is far from perfect, but it does show its possible to do this in realtime, interpret the sketch in 3D, find paths in the sketch and there’s even a little car driving around.

Basically, the program takes something like this:

And turns it into this:

The program is based on an old version of the ARToolkit, coupled with a little bit of openCV for image analysis and OpenSceneGraph for the final assembly and display of the video stream and 3D models. If you want more detailed information on how the ARX program works, or you are interested in other potential applications for this type of program you can view my final paper here. I hear it’s pretty nice.

Download the ARX program here, and an empty marker image (which you need to get started) can be found here. In order to get started, print the image and grab a black marker. Start ARX (make sure a webcam is hooked up) and draw a racetrack in the black square. Basically that’s it. Have fun playing with it