Augmented Reality Assistant

Context-aware AR Assistant using Hololens-2.

Augmented Reality AR Assistant on Hololens-2

  • Full-stack software development of Nakamir Augmented Reality Assistant (NARA).
  • Integrated human body pose estimation using the depth sensors in the Microsoft Hololens-2.
  • Performed camera calibrations and addressed perception pipeline issues.

Skills/Tools: OpenCV, Python, C++, C#, MRTK, Unity, Microsoft Azure.

NARA (Nakamir AR Assistant) records the hand, eye, and head tracking data and spatial point cloud data. Then NARA, plays it back to another user to create a context-aware virtual assistant. As the first engineer of Nakamir, I helped build the entire software stack.
I also helped integrate human body pose estimation on the Microsoft Hololens-2. This resulted in an IEEE publication: https://ieeexplore.ieee.org/abstract/document/10322250.