Thumbnail Image
Publication

Context-Aware Mixed Reality: A Learning-based Framework for Semantic-level Interaction

Chen, Long
; 
Tang, Wen
; 
Zhang, Jian Jun
; 
John, Nigel W.
Advisors
Editors
Other Contributors
Affiliation
Bournemouth University; University of Chester; University of Bradford
EPub Date
Publication Date
2019-11-14
Submitted Date
Collections
Computer Science
Show all metadata
Files
Thumbnail Image
main article
Adobe PDF, 595.81 KB
Other Titles
Abstract
Mixed Reality (MR) is a powerful interactive technology for new types of user experience. We present a semantic-based interactive MR framework that is beyond current geometry-based approaches, offering a step change in generating high-level context-aware interactions. Our key insight is that by building semantic understanding in MR, we can develop a system that not only greatly enhances user experience through object-specific behaviors, but also it paves the way for solving complex interaction design challenges. In this paper, our proposed framework generates semantic properties of the real-world environment through a dense scene reconstruction and deep image understanding scheme. We demonstrate our approach by developing a material-aware prototype system for context-aware physical interactions between the real and virtual objects. Quantitative and qualitative evaluation results show that the framework delivers accurate and consistent semantic information in an interactive MR environment, providing effective real-time semantic level interactions.
Citation
Chen, L., Tang, W., John, N, W., Wan, T, R. & Zhang, J, J. (2019). Context-Aware Mixed Reality: A Learning-based Framework for Semantic-level Interaction. Computer Graphics Forum, 39(1), 484-496.
Publisher
Wiley
Journal
Computer Graphics Forum
Research Unit
URI
http://hdl.handle.net/10034/622767
DOI
10.1111/cgf.13887
PubMed ID
PubMed Central ID
Type
Article
Language
Description
This is the peer reviewed version of the following article: Chen, L., Tang, W., John, N, W., Wan, T, R. & Zhang, J, J. (2019). Context-Aware Mixed Reality: A Learning-based Framework for Semantic-level Interaction. Computer Graphics Forum, which has been published in final form at https://doi.org/10.1111/cgf.13887. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions
Series/Report no.
ISSN
0167-7055
EISSN
1467-8659
ISBN
ISMN
Gov't Doc
Test Link
Sponsors
Additional Links
https://onlinelibrary.wiley.com/journal/14678659