Over the past couple of months I’ve had the chance to get in a bit of Unity research and development time, and one of the areas I have looked at is Augmented Reality. By far the most entertaining experiment so far has been my “AR Driving” demo, which I tried out at the Skive London office recently. It could be said that this particular demo isn’t strictly AR (because it doesn’t overlay graphics on the video stream) but still, it uses AR technology to map the orientation of an AR marker to the steering, acceleration and braking of a car. I recently demoed it in our London office, result: Fun!
The control works by reading the orientation of a single AR marker printed on an A4 sheet. The sheet is folder so that the marker is visible to the camera on one side, and a steering wheel image is visible on the other side – so the user knows which way up to hold the paper!
The rotation around the Z axis (as in, turning the steering wheel) is mapped to the car steering, although not entirely linearly – in order to make the controls easy to use it required a few tweaks, such as smoothing out the sometimes jittery raw data, and giving bias to a ‘not-quite-dead zone’ around the zero-degrees rotation area to make it easier to drive the car in a straight line.
The acceleration and braking controls are implemented by reading the “tilt” of the marker around the X axis. Holding the marker straight-on to the camera gives the equivalent of a steady gentle press on the accelerator. Tilting the top edge towards the camera pushes down the accelerator, while tilting it away releases the accelerator completely and applies the brakes.
The final result is a fairly intuitive driving control, and most people who had a try in the office seemed to find the car responsive and easy to control on their first try, as the video shows!
At the present time (Unity 2.6), implementing AR isn’t entirely straightforward, not least because Unity doesn’t have any built-in way of reading video streams coming in from the webcam, so the webcam stream in this demo is being read via a custom DLL, and the AR markers are picked up from the video stream using the beta version of “UnityAR” (also a custom DLL, which provides an interface to ARToolkit) from Weltenbauer who have kindly made their beta version free. An alternative to this technique is to set up a socket connection from a flash movie, and send either the video data, or the AR marker data via the socket to Unity. Neither of these methods are particularly ideal!
To cut a long story short, because of these custom DLLs the demo is a Windows-only EXE at the moment, and can’t work as a webplayer at all. Now, if UT would add support for webcam input in Unity (in a similar manner to flash) it would open the doors for browser-based AR, which is something that Unity could really excel at.
Unity does have a User Feedback Forum, where you can vote up feature requests for future versions – so if you’re interested in this, vote up Webcam input!