A graphic marker is both the popular metaphor for AR technology (since people associate it with some mysterious symbols printed on paper) and its weakness (since it means that additional equipment is required for AR applications to operate). The need for additional equipment to operate AR applications is often considered their disadvantage, and hinders the popularity of AR games because players can sometimes be lazy and don't want to print out the markers. Nevertheless, the graphic fiducials have very important advantages: first of all, they require less calculation power, and secondly, they are pretty accurate because fiducials are friendly for computer-vision systems and very accessible to perform digital image scanning.
Engineers have tried to "teach" machines to read some printed data by using optical sensors since the end of 1940s. However, the machines were not good at learning human alphabets because doing so would take a long time. Furthermore, the level of errors could be very high. Thus, a new paradigm was invented, that is, writing systems for machines; machine-readable mediums were introduced. Joseph Woodland and Bernard Silver patented the very first barcode in 1952. It looked like a section of a tree with annual growth rings. For the human eye, such an image was a total abstraction, but not for a machine. So, from the very beginning, machine-readable solutions were totally different from any traditional ways to provide information. The data was coded, so humans were not able to read complete information without an automatic translator.
The title of the book encoded in various data codes. All of them were created with a free barcode generator (http://www.racoindustries.com/barcodegenerator/)
Traditional one-dimensional, rectangular barcodes were standardized and became commercially routine in the 1970s. Since then, barcodes have turned into a symbol of communication between industrial machines and the physical world. However, the barcode was not the only one of its kind. Each industry had its own requirements and tasks, so some alternative solutions were introduced as well. First of all, the optical message got a second dimension, so it could store not only a linear array of information, but a matrix of data.
The approach is used in notable products such as PDF417, Data Matrix, MaxiCode, Aztec Code, and of course, the now famous quick response (QR) code, which has become a graphic metaphor of the connection between handheld devices and reality. It's pretty funny because the graphic representation itself is hard to consider beautiful. It looks too pragmatic and foreign, like a robot's fingerprints. The QR code is something that is out. This is because QR code was originally created to track vehicles being manufactured in the automotive factories of Toyota. Nobody had planned for the extra utility of that symbol and only few dreamed about palm-sized computers with cameras and instant Internet connections at the time.
In addition to this, it is interesting to mention that the QR code had a competitor. In 1999, ShotCode was introduced, a data format that was designed for regular cameras that an ordinary consumer might buy, rather than for special industrial systems. It also had a pleasant visual appearance and information inscribed in a circle-based structure; it looked more organic than the traditional barcode. Unfortunately, that was not enough. ShotCode is still in use, but the technology is not so popular. QR codes though, largely through the efforts of the Japanese market, are a sensation now. They are everywhere, in newspapers, magazines, on screens that air TV shows, in video games as hidden secrets, and so on. AR applications are no exception; some of them successfully utilize symbols that look close to QR codes as fiducials. They can be both a source of data (information stored in the data matrix) and a way to display reference points to perform 3D rendering.
Note
Despite the fact that fiducial markers based on simple geometry are very effective, their graphic appearance leaves a lot to be desired. They also reveal all the functional zones of AR gameboards, ruining the magic. To solve this problem, some researchers suggest using invisible markers printed by infrared fluorescent ink, which cannot be perceived by a normal human eye, only by infrared sensitive camera. Thus, infrared fiducial markers can be printed over standard illustrations or text.
The requirements for fiducial markers can be determined very easily. You only need to keep in mind how AR applications work with them in different conditions:
- Two dimensions: Contrary to one-dimensional barcodes, which should be placed either horizontally or vertically in order to be read properly, AR markers can be perceived at any angle. Thus, they should be two-dimensional. This is why in most cases, the markers have square placeholders.
- Lack of rotational symmetry: This property is pretty obvious. For the vision system to easily determine where the horizontal and vertical parts of a marker are, there should be no hint of rotational symmetry. After being rotated to 90 degrees, the angle of the marker should not be equal to that of its initial state. This is why a plain square cannot be a fiducial marker; it must have some asymmetric elements inside.
- Contrast with environment: Markers should be significantly different from the ambiance around them so the image recognition algorithm won't be confused by other patterns or textures. Markers should be like the help symbol in a desert: a triangle with straight lines, which looks very unnatural against the space around it so it cannot be missed by someone looking down at it. As a rule, a contrast-continuous border is required around a marker.
- Achromatic: In most cases, this is not about the chromatic colors. Symbols must be made of black-and-white elements (bi-tonal); image scanning algorithms usually transform images turning any shades into their black-and-white representations. Such design also provides very high visual contrast, guaranteeing that the marker will be perceived even in poor lighting conditions.
- Low density of elements: Despite the fact that cameras in modern handheld devices have a high resolution, patterns inside markers should not include very small elements. Because of bad lighting, distance, distorted perspective, and other factors, the image-recognition system may lose these patterns.
- Diversity: The design should include the opportunity to generate several versions of markers, so the patterns or images inside must be changeable. That will help us use only a few fiducials for a single scene.
One of the major difficulties with fiducial markers is the lack of strong standardization. There is no single format to create geometrical shapes and data coding; each developer of AR platforms tries to invent something unique. Maybe this is quite natural because the technology is pretty young and not that popular yet, so they are only trying to find the right direction. AR technology also gives you some freedom to choose the marker you think is the most effective for your task. Among the popular designs that can be highlighted are as follows:
- ARToolKit markers: These are very popular. They look like thick, square frames with simple contrast symbols inside. The system is notable for its use of Japanese Kanji characters as standard patterns (pretty cyberpunk novel, isn't it?), since ARToolKit was originally created by Dr. Hirokazu Kat from Japan. One common marker displays the Kanji symbol: which looks like a letter "A" without a crossbar and means human. Of course, other types of images can be used, including the Latin alphabet, graphic illustrations, and so on. This is an advantage because it can make the design very human friendly.
- ARTag: This is a more advanced alternative to the traditional ARToolKit markers, with fewer errors and good overall performance. Our system looks like a grid of matrix barcodes in a small dimension (6 x 6 cells); they cover the screen surface like some sort of digital pattern. Because there are a lot of tiny markers, the accuracy of recognizing something that is displayed on the screen is pretty high. If some of the markers are unreadable to a camera because of the environment conditions or some occlusions, the other ones may ensure the process takes place. In some sense, ARTag markers can be considered a cheap alternative to advanced systems that scan surroundings in depth. If an object in the surrounding environment is extensively covered with markers, AR applications will perceive and include them in the improvised virtual mapping. The disadvantage of ARTags is their machine-friendly look, as the design is too technical.
- Siemens Corporate Research (SCR) markers: This system has an attractive simplification as any marker looks like a square array of big dots (4 x 4 elements) and it is very easy to produce, even by drawing them by hand using a marker pen. Besides being logical code, SCR markers look good as dots look pretty harmonic. A similar approach to designing markers is used by the talented indie game developer, int13 (http://int13.net), in their AR projects, and they have proved the efficiency of the markers.
- Canon markers: These are graphic reference points developed by Canon for its head-mounted display called MREAL. The markers consist of a black frame and hexagonal grid of elements inside the display screen. The design looks pretty elegant and according to developers, has few errors.
- reacTIVision markers: One of the most original reference symbol designs is used in an open-source computer vision framework called reacTIVision (http://reactivision.sourceforge.net/). It was developed for the fast tracking of fiducials. The markers have a very unusual, bionic appearance, like that of an amoeba. This geometric shape is not accidental; some topological information is stored inside such markers. Unlike the traditional 2D bar codes, the reacTivision markers stores data tree information. The design is so smart and logical that it helps to remove some stages from the digital-vision algorithms. There are some specially coded vertices and edges inmarkers. That is why the design is so organic and don't use any square shapes. The concept is simply genius! The full story is covered in the article The Design and Evolution of Fiducials for the reacTIVision System by Ross Bencina and Martin Kaltenbrunner (http://mtg.upf.edu/node/442).
The overall list of available marker designs is much longer; for instance, there are fiducials such as the Hoffman marker system (HOM), Institut für Graphische Datenverarbeitung (IGD) markers, and some others. In most cases, their structure is very similar: a thick and contrasting outline and a pattern made of square blocks.
The problem with printing fiducial symbols can be resolved in various creative ways. First of all, the product itself should be so exciting and intriguing that the audience will try their best to find a way to print the marker. Moreover, some types of fiducial markers have a graphic structure that is based on simple geometric forms, for example, SCR or reacTIVision markers may be drawn by hand. Only a piece of paper and a black magic marker are needed. The piece of paper should be taken by a player and applied on screen where a PDF document that contains fiducial markers is shown. Because the paper is translucent, all the shapes can be easily traced. Of course, there can be some inaccuracies, but in many cases, they are irrelevant and the AR works successfully.
Note that if your game uses fiducials, a graphic document with a marker must be uploaded on the homepage of your website. It is also good to have a special test page with the marker displayed clearly so players have the chance to test AR in the game without printing anything, for example, you could use a computer monitor or another mobile device (an iPad can be made to act as a marker). Such practice prevails in the cute game Om Nom: CANDY Flick from ZeptoLab UK Limited (http://www.cuttherope.net/ar/).
The following figure demonstrates concepts of fiducials that don't need a printer and can be done by hand:
I also think that some research should be done about the idea of turning everyday objects into tracking markers. For instance, several flat-headed tacks spread out in specific positions can be turned into a reference point for an AR application. A more elegant solution can be used: a player turns a square sticky note into a fiducial by folding three of its corners, and such approach can be entitled origami AR marker. Another example is stickers or special branded custom cases for the iPhone/iPad, where to play a game, a player must take the case off and put it on the table. The idea is that the virtual game board is always with you. By the way, the silicone iPhone 5C Case with a distinctive grid of holes in it has an interesting potential as an AR accessory; may be somebody could turn such design into an object-based marker.