The Convergence of Internet of Things (IoT) and Mixed Reality (MR) is the emerging trend because the strengths of each technology can complement the weaknesses. IoT has been commercialized for monitoring and remote control, but it is difficult to provide intuitive and user-friendly services due to the abstraction and low immersion of connected objects. On the other hand, MR provides intuitive interaction between digital contents and users, but still has limited service domains such as entertainment and low connectivity with physical devices. The combination of MR with rich immersion related to real world and IoT with ubiquitous connectivity is expected to produce new applications while overcoming limitations. Especially in the industrial domain, field workers wearing MR devices are provided with rich modality which overlays virtual objects on physical objects. Also, it enables collaborative works where remote expert can give instructions to workers and check the progress.
Such a convergence product will be complex and large due to the coexistence of various devices, applications and platforms. Therefore, it requires robust and flexible system design. In this dissertation, we aim to design two novel architectures that complement each other for the convergence of MR and IoT.
In the first detailed study, we propose a novel architecture of a convergence platform that provide connectivity between MR and IoT.
In order to overcome the limitations of existing applications that do not provide domain and device interoperability, the proposed architecture adopts the oneM2M IoT standard as the basic framework. It enables the development of application services used in general-purpose environments without being subordinate to specific domains, and device manufacturers. We implement the proposed architecture utilizing the open-source oneM2M-based IoT server and device and Microsoft HoloLens. We also suggest and demonstrate the practical use cases and discuss the advantages of the proposed architecture.
In the second detailed study, we propose a novel architectural concept called MR Virtual Device (MRVD) that fully provides MR-IoT connectivity and multi-user communication with minimal performance degradation.
MRVD which is a server-side application that replicates MR real device (MRRD) can enhance the performance of MRRD with reducing duplicated data traffic and API calls when multiple traffic offloading runs concurrently.
To demonstrate feasibility and evaluate MRVD, we develop a prototype taking into account of the scenario for the management of fire hydrant cabinets. The prototype utilizes various open-sourced software and commercial MR device, Microsoft HoloLens.