Improving maintenance effi ciency and safety through a human-centric approach

doi:10.1007/s40436-020-00334-x

摘要/Abstract

摘要： This paper presents an adaptive human-machine interface (HMI) that can provide appropriate sets of digital maintenance information and guidance to an operator during maintenance. It takes into consideration the expertise level of the operator and the maintenance context and progress. The proposed human-centric methodology considers the heart rate, intention, and expertise level of the operator, which can be captured using sensors during maintenance. A set of rules is formulated based on the sensor data to infer the state of the operator during a maintenance task. Based on the operator state, the adaptive HMI can augment the operator's senses using a scheme that combines visual, audio, and haptic guidance cues during maintenance to enhance the operator's ability to perceive information and perform maintenance tasks. Various schemes of visual, audio, and haptic cues are developed based on a comparison of the best practices obtained from experienced operators.

The full text can be downloaded at https://link.springer.com/article/10.1007/s40436-020-00334-x

关键词: Augmented reality (AR), Ergonomics, Product development, Maintenance

Abstract: This paper presents an adaptive human-machine interface (HMI) that can provide appropriate sets of digital maintenance information and guidance to an operator during maintenance. It takes into consideration the expertise level of the operator and the maintenance context and progress. The proposed human-centric methodology considers the heart rate, intention, and expertise level of the operator, which can be captured using sensors during maintenance. A set of rules is formulated based on the sensor data to infer the state of the operator during a maintenance task. Based on the operator state, the adaptive HMI can augment the operator's senses using a scheme that combines visual, audio, and haptic guidance cues during maintenance to enhance the operator's ability to perceive information and perform maintenance tasks. Various schemes of visual, audio, and haptic cues are developed based on a comparison of the best practices obtained from experienced operators.

The full text can be downloaded at https://link.springer.com/article/10.1007/s40436-020-00334-x

Key words: Augmented reality (AR), Ergonomics, Product development, Maintenance

C. Y. Siew, S. K. Ong, A. Y. C. Nee. Improving maintenance effi ciency and safety through a human-centric approach[J]. Advances in Manufacturing, 2021, 9(1): 104-114.

参考文献

1. Rødseth H, Schjølberg P, Marhaug A (2017) Deep digital maintenance. Adv Manuf 5:299-310
2. Wang X, Ong SK, Nee AYC (2016) A comprehensive survey of augmented reality assembly research. Adv Manuf 4:1-22
3. Khalek IA, Chalhoub JM, Ayer SK (2019) Augmented reality for identifying maintainability concerns during design. Adv Civil Eng 3:8547928. https://doi.org/10.1155/2019/8547928
4. Gattullo M, Scurati GW, Evangelista A et al (2020) Informing the use of visual assets in industrial augmented reality. In:Proceedings of the international conference on design tools and methods in industrial engineering, 9-10 September, Modena, Italy, pp 106-117
5. Martinettia A, Rajabalinejada M, van Dongena L (2017) Shaping the future maintenance operations:reflections on the adoptions of augmented reality through problems and opportunities. Proc CIRP 59:4-17
6. Abramovici M, Wolf M, Adwernat S et al (2017) Context-aware maintenance support for augmented reality assistance and synchronous multi-user collaboration. Proc CIRP 59:18-22
7. Gattullo M, Dalena V, Evangelista A et al (2019) A contextaware technical information manager for presentation in augmented reality. In:2019 IEEE conference on virtual reality and 3D user interfaces, 23-27 March, Osaka, Japan, pp 939-940
8. Caudell TP, Mizell DW (1992) Augmented reality:an application of heads-up display technology to manual manufacturing processes. In:Proceedings of the 25th international conference on system sciences, Kauai, Hawaii, 7-10 January, pp 659-669
9. Friedrich W, Jahn D, Schmidt L (2002) ARVIKA:augmented reality for development, production and service. In:The 1st international symposium on mixed and augmented reality (ISMAR), Darmstadt, 30 September 1 October, pp 3-4. https://doi.org/10.1109/ismar.2002.1115059
10. De Groote P (1995) Maintenance performance analysis:a practical approach. J Qual Maint Eng 1(2):4-24
11. Dini G, Mura MD (2015) Application of augmented reality techniques in through-life engineering services. Proc CIRP 38:14-23. https://doi.org/10.1016/j.procir.2015.07.044
12. Regenbrecht HT, Specht R (2000) A mobile passive augmented reality device-Mpard. In:ISAR 2000, Munich, Germany, 5-6 October, pp 81-84. https://doi.org/10.1109/isar.2000.880926
13. Ayala GA, Bobadilla IG, Figueroa GA et al (2016) Virtual reality training system for maintenance and operation of high-voltage overhead power lines. Virtual Reality 20(1):1-14. https://doi.org/10.1007/s10055-015-0280-6
14. Jetter J, Eimecke J, Rese A (2018) Augmented reality tools for industrial applications:what are potential key performance indicators and who benefits? Comput Hum Behav 87:18-33
15. Oliveira R, Farinha T, Raposo H et al (2014) Augmented reality and the future of maintenance. In:Proceedings of maintenance performance measurement and management (MPMM) conference, 4-5 September, Coimbra, pp 81-88
16. Gattullo M, Uva AE, Fiorentino M et al (2015) Legibility in industrial AR:text style, color coding, and illuminance. IEEE Comput Gr Appl 35(2):52-61
17. Dünser A, Grasset R, Seichter H et al (2007) Applying HCI principles to AR systems design. In:Workshop proceedings of mixed reality user interfaces:specification, authoring, adaptation (MRUI'07), 11 Mar, Charlotte, NC, USA, pp 37-42
18. Müller T (2015) Towards a framework for information presentation in augmented reality for the support of procedural tasks. In:International conference on augmented and virtual reality, 31 August 3 September, Lecce, Italy, pp 490-497
19. Aschenbrenner D, Latoschik ME, Schilling K (2016) Industrial maintenance with augmented reality:two case studies. In:Proceedings of the 22nd ACM conference on virtual reality software and technology (VRST16). ACM Press, Garching, Germany, pp 341-342
20. Mourtzisa D, Zogopoulos V, Vlachou E (2017) Augmented reality application to support remote maintenance as a service in the robotics industry. Proc CIRP 63:46-51
21. Kammerer K, Pryss R, Sommer K et al (2018) Towards contextaware process guidance in cyber-physical systems with augmented reality. In:The 4th international workshop on requirements engineering for self-adaptive, collaborative, and cyber physical systems, 20 Auguest, Banff, Canada, pp 44-51
22. Zhu J, Ong SK, Nee AYC (2015) A context-aware augmented reality assisted maintenance system. Int J Comput Integr Manuf 28(2):213-225
23. Erkoyuncu JA, del Amo IF, Dalle MM et al (2017) Improving efficiency of industrial maintenance with context aware adaptive authoring in augmented reality. Ann CIRP 66:465-468
24. Hervás R, Bravo J, Fontecha J et al (2013) Achieving adaptive augmented reality through ontological context-awareness applied to AAL scenarios. J Univ Comput Sci 19(9):1334-1349
25. Holm M, Danielsson O, Syberfeldt A et al (2017) Adaptive instructions to novice shop-floor operators using augmented reality. J Ind Prod Eng 34(5):362-374
26. Syberfeldt A, Danielsson O, Holm M et al (2016) Dynamic operator instructions based on augmented reality and rule-based expert systems. Proc CIRP 41:346-351
27. Lages WS, Bowman DA (2019) Walking with adaptive augmented reality workspaces:design and usage patterns. In:The 24th international conference on intelligent user interfaces, 17-20 March, CM, New York, USA, pp 356-366
28. Petrangeli S, Simon G, Wang H et al (2019) Dynamic adaptive streaming for augmented reality applications. In:IEEE international symposium on multimedia (ISM), 9-11 December, San Diego, California, USA, pp 56-63
29. Tsai CH, Huang JY (2018) Augmented reality display based on user behavior. Comput Stand Interfaces 55:171-181
30. Healey JA, Picard RW (2005) Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans Intelligent Transp Syst 6(2):156-166
31. Knopfle C, Weidenhausen J, Chauvigne L et al (2005) Template based authoring for AR based service scenarios. In:IEEE proceedings of virtual reality, 12-16 March, Bonn, Germany
32. Gattullo M, Uva AE, Fiorentino M et al (2017) From paper manual to AR manual:do we still need text? Proc Manuf 11:1303-1310
33. Masoni R, Ferrise F, Bordegoni M et al (2017) Supporting remote maintenance in Industry 4.0 through augmented reality. Proc Manuf 11:1296-1302
34. Jo GS, Oh KJ, Ha I et al (2014) A unified framework for augmented reality and knowledge-based systems in maintaining aircraft. In:Proceedings of the national conference on artificial intelligence, 27-31 July, Quebec, Canada, pp 2990-2997
35. Palmarini R, Erkoyuncu JA, Roy R et al (2018) A systematic review of augmented reality applications in maintenance. Robot Comput Integr Manuf 49:215-228

[1]	S. K. Ong, X. Wang, A. Y. C. Nee. 3D bare-hand interactions enabling ubiquitous interactions with smart objects[J]. Advances in Manufacturing, 2020, 8(2): 133-143.
[2]	S. K. Ong, A. Y. C. Nee, A. W. W. Yew, N. K. Thanigaivel. AR-assisted robot welding programming[J]. Advances in Manufacturing, 2020, 8(1): 40-48.
[3]	Harald Rødseth, Per Schjølberg, Andreas Marhaug. Deep digital maintenance[J]. Advances in Manufacturing, 2017, 5(4): 299-310.
[4]	Zhe Li, Yi Wang, Ke-Sheng Wang. Intelligent predictive maintenance for fault diagnosis and prognosis in machine centers: Industry 4.0 scenario[J]. Advances in Manufacturing, 2017, 5(4): 377-387.
[5]	D. Ni, A. W. W. Yew, S. K. Ong, A. Y. C. Nee. Haptic and visual augmented reality interface for programming welding robots[J]. Advances in Manufacturing, 2017, 5(3): 191-198.
[6]	X. Wang, S. K. Ong, A. Y. C. Nee. A comprehensive survey of augmented reality assembly research[J]. Advances in Manufacturing, 2016, 4(1): 1-22.
[7]	Ke-Sheng Wang, Zhe Li, Jørgen Braaten, Quan Yu. Interpretation and compensation of backlash error data in machine centers for intelligent predictive maintenance using ANNs[J]. Advances in Manufacturing, 2015, 3(2): 97-104.
[8]	Harald Rødseth, Terje Skarlo, Per Schjølberg. Profit loss indicator: a novel maintenance indicator applied for integrated planning[J]. Advances in Manufacturing, 2015, 3(2): 139-150.
[9]	H. C. Fang, S. K. Ong, A. Y. C. Nee. Novel AR-based interface for human-robot interaction and visualization[J]. Advances in Manufacturing, 2014, 2(4): 275-288.
[10]	Alemu Moges Belay ? Torgeir Welo ?Petri Helo. Approaching lean product development using system dynamics:investigating front-load effects[J]. Advances in Manufacturing, 2014, 2(2): 130-140.
[11]	Ming Chen Jin-Fei Liu. Virtual simulation of production line for ergonomics evaluation[J]. Advances in Manufacturing, 2014, 2(1): 48-53.