3D bare-hand interactions enabling ubiquitous interactions with smart objects

doi:10.1007/s40436-020-00295-1

Advances in Manufacturing ›› 2020, Vol. 8 ›› Issue (2): 133-143.doi: 10.1007/s40436-020-00295-1

• ARTICLES • 下一篇

3D bare-hand interactions enabling ubiquitous interactions with smart objects

S. K. Ong, X. Wang, A. Y. C. Nee

Department of Mechanical Engineering, Faculty of Engineering, National University of Singapore, Singapore

收稿日期:2019-03-22 修回日期:2019-11-14 出版日期:2020-06-25 发布日期:2020-06-08
通讯作者: S. K. Ong E-mail:mpeongsk@nus.edu.sg
基金资助:
This research was supported by the Singapore A*STAR Agency for Science, Technology and Research Public Sector Research Funding Programme (Grant No. 1521200081).

3D bare-hand interactions enabling ubiquitous interactions with smart objects

S. K. Ong, X. Wang, A. Y. C. Nee

Department of Mechanical Engineering, Faculty of Engineering, National University of Singapore, Singapore

Received:2019-03-22 Revised:2019-11-14 Online:2020-06-25 Published:2020-06-08
Contact: S. K. Ong E-mail:mpeongsk@nus.edu.sg
Supported by:
This research was supported by the Singapore A*STAR Agency for Science, Technology and Research Public Sector Research Funding Programme (Grant No. 1521200081).

摘要/Abstract

摘要： Ubiquitous augmented reality (UAR) implementation can benefit smart shop floor operations significantly. UAR from a user's first-person view can support and provide the user with suitable and comprehensive information without him/her being distracted from ongoing tasks. A natural hand-based interaction interface, namely, a mobile bare-hand interface (MBHI), is proposed to assist a user in exploring and navigating a large amount of information for a task in the user's first-person view. The integration of a smart shop floor and UAR-based MBHI is particularly challenging. A real shop floor environment is composed of challenging conditions for the implementation of UAR, e.g., messy backgrounds and significant changes in illumination conditions. Meanwhile, the MBHI is required to provide precise and quick responses to minimize the difficulty of a user's task. In this study, a wearable UAR system integrated with an MBHI is proposed to augment the shop floor environment with smart information. A case study is implemented to demonstrate the practicality and effectiveness of the proposed UAR and MBHI system.

The full text can be downloaded at https://link.springer.com/article/10.1007/s40436-020-00295-1

关键词: Augmented reality (AR), Mobile bare-hand interaction (MBHI), Smart objects (SOs), Manufacturing shop-floor

Abstract: Ubiquitous augmented reality (UAR) implementation can benefit smart shop floor operations significantly. UAR from a user's first-person view can support and provide the user with suitable and comprehensive information without him/her being distracted from ongoing tasks. A natural hand-based interaction interface, namely, a mobile bare-hand interface (MBHI), is proposed to assist a user in exploring and navigating a large amount of information for a task in the user's first-person view. The integration of a smart shop floor and UAR-based MBHI is particularly challenging. A real shop floor environment is composed of challenging conditions for the implementation of UAR, e.g., messy backgrounds and significant changes in illumination conditions. Meanwhile, the MBHI is required to provide precise and quick responses to minimize the difficulty of a user's task. In this study, a wearable UAR system integrated with an MBHI is proposed to augment the shop floor environment with smart information. A case study is implemented to demonstrate the practicality and effectiveness of the proposed UAR and MBHI system.

The full text can be downloaded at https://link.springer.com/article/10.1007/s40436-020-00295-1

Key words: Augmented reality (AR), Mobile bare-hand interaction (MBHI), Smart objects (SOs), Manufacturing shop-floor

S. K. Ong, X. Wang, A. Y. C. Nee. 3D bare-hand interactions enabling ubiquitous interactions with smart objects[J]. Advances in Manufacturing, 2020, 8(2): 133-143.

参考文献

1. Monostori L, Kadar B, Bauernhansl T et al (2016) Cyber-physical systems in manufacturing. CIRP Ann Manuf Technol 65(2):621-641
2. Lee CKM, Zhang SZ, Ng KKH (2017) Development of an industrial Internet of things suite for smart factory towards reindustrialization. Adv Manuf 5:335-343
3. Yew AWW, Ong SK, Nee AYC (2016) Towards a griddable distributed manufacturing system with augmented reality interfaces. Robot Comput Integr Manuf 39:43-55
4. Voinov N, Chernorutsky I, Drobintsev P et al (2017) An approach to net-centric control automation of technological processes within industrial IoT systems. Adv Manuf 5:388-393
5. Kortuem G, Kawsar F, Sundrammorthy V (2010) Smart objects as building blocks for the internet of things. IEEE Internet Comput 14(1):44-51
6. Burmeister D, Burmann F, Schrader A (2017) The smart object description language:modeling interaction capabilities for selfreflection. In:Proceedings of 2017 IEEE international conference on pervasive computing and communications workshops, pp 503-508
7. Grubert J, Langlotz T, Zollmann S et al (2016) Towards pervasive augmented reality:context-awareness in augmented reality. IEEE Trans Vis Comput Graph 23(6):1706-1724
8. Wang X, Ong SK, Nee AYC (2016) A comprehensive survey of augmented reality assembly research. Adv Manuf 4(1):1-22
9. Teyssier M, Cliquet G, Richir S (2016) ArLive:unified approach of interaction between users, operable space and smart objects. In:Proceedings of the 2016 virtual reality international conference, ACM
10. Wirtz H, Ruth J, Serror M et al (2015) Enabling ubiquitous interaction with smart things. In:Proceedings of 2015 12th annual IEEE international conference on sensing, communication, and networking, pp 256-264
11. Wang X, Yew AWW, Ong SK et al (2019) Enhancing smart shop floor management with ubiquitous augmented reality. Int J Prod Res. https://doi.org/10.1080/00207543.2019.1629667
12. Sollich H, von Zadow U, Pietzsch T et al (2016) Exploring timedependent scientific data using spatially aware mobiles and large displays. In:Proceedings of the 2016 ACM on interactive surfaces and spaces, pp 349-354
13. Besancon L, Issartel P, Ammi M et al (2017) Hybrid tactile/tangible interaction for 3D data exploration. IEEE Trans Vis Comput Graph 23(1):881-890
14. Chong JWS, Ong SK, Nee AYC (2007) Methodologies for immersive robot programming in an augmented reality environment. Int J Virtual Real 6(1):69-79
15. Krichenbauer M, Yamamoto G, Taketomi T et al (2014) Towards augmented reality user interfaces in 3D media production. In:Proceedings of 2014 IEEE international symposium on mixed and augmented reality (ISMAR 2014), pp 23-28
16. Feng L, Yang X, Xiao S (2017) MagicToon:a 2D-to-3D creative cartoon modeling system with mobile AR. In:Proceedings of 2017 IEEE virtual reality, pp 195-204
17. Wang X, Ong SK, Nee AYC (2016) Real-virtual components interaction for assembly simulation and planning. Robot Comput Integr Manuf 41:102-114
18. Wang X, Ong SK, Nee AYC (2016) Multi-modal augmentedreality assembly guidance based on bare-hand interface. Adv Eng Inf 30(3):406-421
19. Arroyave-Tobon S, Osorio-Gomez G, Cardona-McCormick JF (2015) Air-modeling:a tool for gesture-based solid modelling in context during early design stages in AR environments. Comput Ind 66:73-81
20. Lin S, Cheng HF, Li W et al (2017) Ubii:physical world interaction through augmented reality. IEEE Trans Mobile Comput 16(3):872-885
21. Korinth M, Sommer-Dittrich T, Reichert M et al (2019) Design and evaluation of a virtual reality-based car configuration concept. In:CVC 2019 advances in computer vision, pp 169-189
22. Velaz Y, Arce JR, Gutierrez T et al (2014) The influence interaction technology on the learning of assembly tasks using virtual reality. J Comput Inf Sci Eng 14(4):041007
23. Valentini PP (2009) Interactive virtual assembling in augmented reality. Int J Interact Des Manuf 3(2):109-119
24. Dhawale P, Masoodian M, Rogers B (2006) Bare-hand 3D gesture input to interactive systems. In:Proceedings of the 7th international conference on NZ Chapter of the ACM's SIG on human-computer interaction, vol 158, pp 25-32
25. Wang RY, Paris S, Popovic J (2011) 6D hands:Markerless hand tracking for computer aided design. In:Proceedings of the 24th annual ACM symposium on user interface software and technology, pp 549-557
26. Hilliges O, Izadi S, Wilson AD et al (2009) Interactions in the air:adding further depth to interactive tabletops. In:Proceedings of the 22nd annual ACM symposium on user interface software and technology, pp 139-148
27. Kojima Y, Yasumuro Y, Sasaki H et al (2001) Hand manipulation of virtual objects in wearable augmented reality. In:Proceedings of the seventh international conference on virtual systems and multimedia, pp 463-469
28. Erkoyuncu JA, del Amo IF, Mura MD et al (2017) Improving efficiency of industrial maintenance with context aware adaptive authoring in augmented reality. CIRP Ann Manuf Technol 66(1):465-468
29. Mourtzis D, Vlachou A, Zogopoulos V (2017) Cloud-based augmented reality remote maintenance through shop-floor monitoring:a product-service system approach. J Manuf Sci Eng 139(6):061011
30. Bergesio L, Bernardos AM, Casar JR (2017) An object-oriented model for object orchestration in smart environments. Proc Comput Sci 109:440-447
31. Yachir A, Djamaa B, Mecheti A et al (2016) A comprehensive semantic model for smart object description and request resolution in the internet of things. Proc Comput Sci 83:147-154
32. Hu W, Sun Y, Li X et al (2016) Cam-shift target tracking approach based on back projection method with distance weights. In:Proceedings of 2016 IEEE international conference on wavelet analysis and pattern recognition (ICWAPR), pp 252-257
33. https://opencv.org. Accessed on Oct 19 2017
34. Rautaray SS, Agrawal A (2015) Vision based hand gesture recognition for human computer interaction:a survey. Artif Intell Rev 43(1):1-54
35. Shen Y, Ong SK, Nee AYC (2011) Vision-based hand interaction in augmented reality environment. Int J Hum Comput Interact 27(7):523-544

[1]	S. K. Ong, A. Y. C. Nee, A. W. W. Yew, N. K. Thanigaivel. AR-assisted robot welding programming[J]. Advances in Manufacturing, 2020, 8(1): 40-48.
[2]	D. Ni, A. W. W. Yew, S. K. Ong, A. Y. C. Nee. Haptic and visual augmented reality interface for programming welding robots[J]. Advances in Manufacturing, 2017, 5(3): 191-198.
[3]	X. Wang, S. K. Ong, A. Y. C. Nee. A comprehensive survey of augmented reality assembly research[J]. Advances in Manufacturing, 2016, 4(1): 1-22.
[4]	H. C. Fang, S. K. Ong, A. Y. C. Nee. Novel AR-based interface for human-robot interaction and visualization[J]. Advances in Manufacturing, 2014, 2(4): 275-288.

3D bare-hand interactions enabling ubiquitous interactions with smart objects

3D bare-hand interactions enabling ubiquitous interactions with smart objects

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 4

编辑推荐

Metrics

本文评价