Augmented Reality 101

Augmented Reality (AR) is a "Mixed Reality" technology where the virtual and real worlds appear together in a real time system. An augmented reality system generates a composite view for the user that is a combination of the real scene viewed by the user and a virtual scene generated by the computer that augments the scene with additional information. This technology of movies like Minority Report is no longer science fiction, but has been for a while science fact.

Even though AR has been developed in research labs and used by the military (Office of Naval Research) for decades, it is only until recently that hardware and software is making it more accessible for the mainstream. In order to superimpose graphics over a real environment in real-time, one needs mobile computing, a tracking or identification system, and a display. Now with more powerful cell phones, Iphone, and other hand-held computing devices, AR is primed to move into the mainstream. Google's Android phone operating system already has several AR applications being developed for it like Enkin, an application that combines GPS, orientation sensors, 3d graphics, live video, and other technologies for navigation.

The NFL has used the "First Down Line" for a number of seasons now. Various institutes have begun using it for cultural awareness and history education. Metaio, a maker of various AR software products, recently showcased AR pop-up books at the Frankfurt Book Fair in Germany much like the book in this video. Construction, medicine, gaming, and a growing number of industries are taking advantage of this ability to overlay information, 3D graphics, animation, sound, and virtual over the real world.

Here is an example of an application used by BMW Research. A great example of how this technology can be used to enhance education and training.


Eye of Judgement, a video game made for the Playstation 3 uses the PS Eye camera, and a form of CyberCode to allow gamers the chance to play an AR battle trading card game. CyberCode (developed by Dr. Jun Rekimoto in 1996) is a visual tagging system designed for AR applications, based on an image recognition technology that uses visual-markers. To recognize the code, a captured image is binarized and candidate areas are picked up. If the black and white cell pattern in the marker area is the correct bit pattern, the area is recognized as a correct CyberCode, just like a normal bar code. While recognizing the 2D codes, the 3D position and orientation of the camera is estimated in relation to the distortion of the incoming visual code images. Using CyberCode and 2D markers like it, eliminates the need for expensive 3D position sensor devices. All it requires is a low-cost camera, visual-markers that can be printed on regular paper, and an AR program that can read the markers.

I am currently setting up my own augmented reality system using the ARToolKit system developed by the Human Interface Technology Lab at Washington University. ARToolKit is a C and C++ language software library that lets programmers easily develop Augmented Reality applications. Here are some examples of how it is being used. This system uses markers similar to CyberCode for identifying 3D files, can be downloaded with just OpenGL graphics support, and can support VRML models and animations with the OpenVRML renderer download. A glimpse at what I have dabbled in so far using the ARTag system.



I am always amazed at seeing what cool applications the adopters and developers of this technology are building so expect more on my blog about AR. I personally am working with others on some cool projects that include this enhancing technology so stay tuned. As always for those interested, send me a mail for more information. AR is bringing the mixed reality experience known as the Metaverse ever closer, and is changing the way we live, learn, and interact with information, others, and the world we live in.