General Motors is one of the top automotive manufacturers, selling over 6 million vehicles annually worldwide. GM is headquartered in Detroit, Michigan, and is known for brands such as GMC, Buick, Chevrolet and Cadillac.

As augmented reality becomes more accessible to large companies, new ways of utilizing this technology emerge, from remote training to visualizing 3D models in real time. GM has explored many possible corporate implementations of augmented reality, yet one unexplored area is augmented reality interacting with hardware.

Our Augmented Reality Utilizing IoT (Internet of Things) Technology is an augmented reality headset that facilitates virtual interaction with the physical world. Our system employs the headset as a replacement for a physical key allowing access into restricted areas and content.

When a user needs access to a locked container, they put on the headset and open our application. A holographic button is displayed directly in front of the user that shows the status of the physical lock.

Once the button is pressed, the lock receives a signal to unlock, and the holographic button reflects this new status. Inside the container is a physical QR code that the headset scans. This QR code generates a hologram in the visual field of the headset to instruct the user in completing their task (shown on the right).

Our software integrates augmented reality with physical devices and demonstrates the viability of augmented reality and its uses for General Motors.

Our front end consists of an Unreal Engine 4.27.2 program running on a Microsoft HoloLens 2. The back end is a Flask server hosted on a Raspberry Pi. The server controls a Servo motor that is connected to a sliding deadbolt.