One of the big questions in AR is how to interact with the environment. Wearing AR glasses allows you to see virtual images projected into the real world. But how does one interact with the virtual images? One way is with a virtual touch screen. This can be done in a number of ways. One option is to have a camera which tracks the movement and identifies that user's hands and the hand locations in 3D space. While this sounds good in principle, there are several real world challenges with this. First, identifying the hand as the user's hand and not someone else's hand can be difficult. And second, tracking the exact hand position and finger tip location requires very sophisticated image processing which may be too processor intensive for the CPU inside the AR glasses. And third, interacting in 3D space, essentially with air, gives no tactile or haptic feedback. Waving your hand or fake-pressing in air can be frustrating because your mind is seeing something but your hands are clearly telling you that it does not exist. For a full immersive effect, this is not ideal.

A more cost effective way to do it would be to have a touch pad which is the blank canvas for the AR environment. The touch pad could have a special overlay on it which would be identified by the AR glasses and allow the glasses to project an image on its boundaries. For example, the touchpad has a special ivory colored overlay which is easily identified by the glasses. The glasses then project an image on top of the touchpad. In this case you can now touch the touchpad, get haptic feedback that you are touching an actual object, and the touchpad can track position of the finger(s) and report that back to the AR glasses to allow the proper interaction.

The touchpad also does not need to be a simple rectangle touchpad. It can be anywhere, it can be on a device. For example, a controller or gun could have a touch pad anywhere on it and when the user looks at that location, the glasses could project something on that area that the user can interact with. Touch sensitive fabrics can be used to add a virtual control panel to a user's sleeve.

The touchpad could be wired to the glasses, wired to a PC, or wireless with something like bluetooth.

In general a touchpad in an AR environment could just what the user needs in order to interact with anything required!