Advances in Manufacturing ›› 2024, Vol. 12 ›› Issue (4): 764-783.doi: 10.1007/s40436-023-00479-5
• • 上一篇
Clara Garcia, Mario Ortega, Eugenio Ivorra, Manuel Contero, Pau Mora, Mariano L. Alcañiz
收稿日期:2023-05-08
修回日期:2023-10-16
发布日期:2024-12-06
通讯作者:
Eugenio Ivorra,E-mail:euivmar@upvnet.upv.es
E-mail:euivmar@upvnet.upv.es
作者简介:Clara Garcia is currently working as a researcher at the HumanTech institute, where she is focused on topics related to 3D scene understanding, industrial augmented and mixed reality, and artificial intelligence. She studied a Bachelor’s degree in Audiovisual Systems Engineering at Pompeu Fabra University in Barcelona. Later on, she pursued a Master’s degree in Computer Vision at the Autonomous University of Barcelona. Throughout her professional career, she has also worked at a technological centre developing computer vision solutions for satellite images. She has also worked in the light field image sector, applying image processing and computer vision approaches. Mario Ortega earned his Mathematics degree from the University of Valencia and further pursued a Diploma of Advanced Studies (DEA) in Applied Mathematics at the same institution. In 2009, he obtained a PhD in Computer Science from the Polytechnic University of Valencia. With over 20 years of research and development experience, Dr. Ortega has honed his expertise in Computer Vision, applying it to fields such as Augmented Reality and Medical Imaging. In 2004, Dr. Ortega joined Human-Tech, where he contributed to various medical image analysis projects related to neurosurgery. He now spearheads the Human-Tech scientific team, which concentrates on Augmented Reality, Computer Vision, and Natural User Interfaces. Their efforts are particularly focused on diverse applications within the industrial sector. Dr. Ortega has demonstrated a strong aptitude for securing and managing international and national research projects and has published numerous scientific articles in prestigious journals and international conferences. Eugenio Ivorra holds a Bachelor’s degree in Computer Science and a Master’s degree in Automatics and Industrial Computing, both from the Universitat Politècnica de València (UPV). He also obtained his Ph.D. in Automatics, Robotics and Industrial Computing from the same university in 2015. He started his research career in 2009 at the Institute of Automatics and Industrial Computing of the UPV, until 2016 when he moved to the Institute of Biomedical Research and Innovation where he currently holds the position of Senior Research Technician with a Ph.D. degree. His main research areas focus on computer vision, hyperspectral vision, and augmented reality, specifically in the detection and estimation of the 3D pose of objects. In these research areas, he has published 21 highly relevant JCR scientific articles. He has also participated in 13 RD projects both nationally and internationally. Manuel Contero is a Full Professor in the Department of Graphic Engineering at the Polytechnic University of Valencia (UPV). He received his degree in Industrial Engineering from the UPV in 1990, and completed his Ph.D. studies at the same university in 1995. He joined the university Jaume I of Castellón in 1993 as a Faculty Assistant and later as a full-time Associate Professor, before being promoted to Full Professor at the UPV in 2008. His research areas focus on collaborative engineering, human-machine interaction, sketch-based modeling, the development of spatial skills, and the application of new technologies in education and technical training. He has directed 13 doctoral theses and supervised 59 final degree projects. He has published over 100 articles in scientific journals and participated in over 150 scientific congresses. He has also been the Principal Investigator of four National RD projects and has participated in several competitive European, national, and regional projects. Pau Mora is a graduate of Industrial Electronics and Automation Engineering from the Technical University of Valencia (UPV) since 2021 and one year later obtained a Master’s Degree in Automation and Industrial Computing at the same university. Currently, he is pursuing a Ph.D. in Automation, Robotics, and Industrial Computer Science at UPV. And working as a researcher at the Human-Tech Institute, where he is developing his doctoral thesis focused on computer vision and augmented reality. Mariano L. Alcañiz is the founding director of the Immersive Neurotechnologies Laboratory (LabLENI) at the UPV and a Full Professor at the Polytechnic University of Valencia. His research interests revolve around a better understanding and improvement of human cognition, combining knowledge and methods from computer science, psychology, and neuro-science. His work focuses on using empirical methodologies from behavioral sciences to explore people while they interact in these digital worlds, but also on research aimed at developing new ways to produce extended reality (XR) simulations. He has published over 250 academic articles, in interdisciplinary journals such as Scientific Reports and PLoS One, as well as in specific domain journals in the fields of engineering, computer science, psychology, marketing, management, and education. His work has been continuously funded by the Spanish Research Agency and the European Commission for 30 years. He has been part of several Spanish and EU scientific committees investigating the use of ICT in different disciplines. He has advised on Virtual Reality policy projects for various government agencies and companies and founded several spin-off companies related to his field of research, such as Previ and Quatechnion.
基金资助:Clara Garcia, Mario Ortega, Eugenio Ivorra, Manuel Contero, Pau Mora, Mariano L. Alcañiz
Received:2023-05-08
Revised:2023-10-16
Published:2024-12-06
Contact:
Eugenio Ivorra,E-mail:euivmar@upvnet.upv.es
E-mail:euivmar@upvnet.upv.es
Supported by:摘要： During the last two decades, industrial applications of augmented reality (AR) have been incorporated in sectors such as automotive or aeronautics in tasks including manufacturing, maintenance, and assembly. However, AR’s potential has yet to be demonstrated in the railway sector due to its complexity and difficulties in automating tasks. This work aims to present an AR system based on HoloLens 2 to assist the assembly process of insulation panels in the railway sector significantly decreasing the time required to perform the assembly. Along with the technical description of the system, an exhaustive validation process is provided where the assembly using the developed system is compared to the traditional assembly method as used by a company that has facilitated a case study. The results obtained show that the system presented outperforms the traditional solution by 78% in the time spent in the localization subtask, which means a 47% decrease in the global assembly time. Additionally, it decreases the number of errors in 88% of the cases, obtaining a more precise and almost error-free assembly process. Finally, it is also proven that using AR removes the dependence on users’ prior knowledge of the system to facilitate assembly.
The full text can be downloaded at https://link.springer.com/article/10.1007/s40436-023-00479-5
Clara Garcia, Mario Ortega, Eugenio Ivorra, Manuel Contero, Pau Mora, Mariano L. Alcañiz. Holorailway: an augmented reality system to support assembly operations in the railway industry[J]. Advances in Manufacturing, 2024, 12(4): 764-783.
Clara Garcia, Mario Ortega, Eugenio Ivorra, Manuel Contero, Pau Mora, Mariano L. Alcañiz. Holorailway: an augmented reality system to support assembly operations in the railway industry[J]. Advances in Manufacturing, 2024, 12(4): 764-783.
| [1] Schwab K (2017) The fourth industrial revolution, Hardcover edn, p. 192. Currency, Geneva [2] Lasi H, Fettke P, Kemper HG et al (2014) Industry 4.0. Bus Inf Syst Eng 6(4):239-242 [3] Grieves M (2014) Digital twin: manufacturing excellence through virtual factory replication. White Pap 1:1-7 [4] Qi Q, Tao F (2018) Digital twin and big data towards smart manufacturing and Industry 4.0: 360 degree comparison. IEEE Access 6:3585-3593 [5] Carmigniani J, Furht B (2011) Augmented reality: an overview. In: Furht B (eds) Handbook of augmented reality. Springer, New York, pp 3-46 [6] De Pace F, Manuri F, Sanna A (2018) Augmented reality in Industry 4.0. Am J Comput Sci Inf Technol 6(1):1-7 [7] Santi GM, Ceruti A, Liverani A et al (2021) Augmented reality in Industry 4.0 and future innovation programs. Technologies 9(2):33. https://doi.org/10.3390/technologies9020033 [8] Bongomin O, Yemane A, Kembabazi B et al (2020) Industry 4.0 disruption and its neologisms in major industrial sectors: a state of the art. J Eng 2020:66. https://doi.org/10.1155/2020/8090521 [9] Langley A, Lawson G, Hermawati S et al (2016) Establishing the usability of a virtual training system for assembly operations within the automotive industry. Hum Factors Ergon Manuf Serv Ind 26(6):667-679 [10] Fraga-Lamas P, Fernández-Caramés TM, Blanco-Novoa O et al (2018) A review on industrial augmented reality systems for the Industry 4.0 shipyard. IEEE Access 6:13358-13375 [11] Regenbrecht H, Baratoff G, Wilke W (2005) Augmented reality projects in the automotive and aerospace industries. IEEE Comput Graph Appl 25(6):48-56 [12] Popov O, Iatsyshyn A, Sokolov D et al (2021) Application of virtual and augmented reality at nuclear power plants. In: Zaporozhets A, Artemchuk V (eds) Systems, decision and control in energy II, Springer, Cham, pp 243-260 [13] de Souza CLF, Mariano FCMQ, Zorzal ER (2020) A survey of industrial augmented reality. Comput Ind Eng 139:106159. https://doi.org/10.1016/j.cie.2019.106159 [14] Geng J, Song X, Pan Y et al (2020) A systematic design method of adaptive augmented reality work instruction for complex industrial operations. Comput Ind 119:103229. https://doi.org/10.1016/j.compind.2020.103229 [15] Dalle Mura M, Dini G (2021) Augmented reality in assembly systems: state of the art and future perspectives. In: International precision assembly seminar. Springer, pp 3-22 [16] Ashwini KB, Patil PN et al (2020) Tracking methods in augmented reality-explore the usage of marker-based tracking. In: Proceedings of the 2nd international conference on IoT, social, mobile, analytics & cloud in computational vision & bio-engineering (ISMAC-CVB 2020) [17] Masood T, Egger J (2019) Augmented reality in support of Industry 4.0—implementation challenges and success factors. Robot Comput Integr Manuf 58:181-195 [18] Georgel PF (2011) Is there a reality in industrial augmented reality? In: 2011 10th IEEE international symposium on mixed and augmented reality. IEEE, pp 201-210 [19] Ortega M, Ivorra E, Juan A et al (2021) Mantra: an effective system based on augmented reality and infrared thermography for industrial maintenance. Appl Sci 11(1):385. https://doi.org/10.3390/app11010385 [20] Ungureanu D, Bogo F, Galliani S et al (2020) Hololens 2 research mode as a tool for computer vision research. arXiv preprint arXiv:2008.11239 [21] Vidal-Balea A, Blanco-Novoa O, Fraga-Lamas P et al (2020) Creating collaborative augmented reality experiences for Industry 4.0 training and assistance applications: performance evaluation in the shipyard of the future. Appl Sci 10(24):9073. https://doi.org/10.3390/app10249073 [22] Psarommatis F, May G, Dreyfus PA et al (2020) Zero defect manufacturing: state-of-the-art review, shortcomings and future directions in research. Int J Prod Res 58(1):1-17 [23] Psarommatis F, Sousa J, Mendonça JP et al (2022) Zero-defect manufacturing the approach for higher manufacturing sustain-ability in the era of Industry 4.0: a position paper. Int J Prod Res 60(1):73-91 [24] Wohlgemuth W, Triebfürst G (2000) Arvika: augmented reality for development, production and service. In: Proceedings of DARE 2000 on designing augmented reality environments, pp 151-152 [25] Lima JP, Roberto R, Simões F et al (2017) Markerless tracking system for augmented reality in the automotive Industry. Expert Syst Appl 82:100-114 [26] Egger J, Masood T (2020) Augmented reality in support of intelligent manufacturing—a systematic literature review. Comput Ind Eng 140:106195. https://doi.org/10.1016/j.cie.2019.106195 [27] Oh YJ, Park KY, Kim EK (2014) Mobile augmented reality system for design drawing visualization. In: 16th International conference on advanced communication technology. IEEE, pp 1296-1300 [28] Masoni R, Ferrise F, Bordegoni M et al (2017) Supporting remote maintenance in Industry 4.0 through augmented reality. Procedia Manuf 11:1296-1302 [29] Funk M, Kosch T, Schmidt A (2016) Interactive worker assistance: comparing the effects of in-situ projection, head-mounted displays, tablet, and paper instructions. In: Proceedings of the 2016 ACM international joint conference on pervasive and ubiquitous computing, pp 934-939 [30] Davies R (2015) Industry 4.0: digitalisation for productivity and growth. EPRS: European Parliamentary Research Service. https://policycommons.net/artifacts/1335939/industry-40/1942749/ [31] Masood T, Egger J (2020) Adopting augmented reality in the age of industrial digitalization. Comput Ind 115:103112. https://doi.org/10.1016/j.compind.2019.07.002 [32] Doshi A, Smith RT, Thomas BH et al (2017) Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing. Int J Adv Manuf Technol 89(5):1279-1293 [33] Mourtzis D, Zogopoulos V, Vlachou E (2018) Augmented reality supported product design towards Industry 4.0: a teaching factory paradigm. Procedia Manuf 23:207-212 [34] Alvarez H, Aguinaga I, Borro D (2011) Providing guidance for maintenance operations using automatic markerless augmented reality system. In: 2011 10th IEEE international symposium on mixed and augmented reality. IEEE, pp 181-190 [35] Fiorentino M, Uva AE, Gattullo M et al (2014) Augmented reality on large screen for interactive maintenance instructions. Comput Ind 65(2):270-278 [36] Mourtzis D, Zogopoulos V, Vlachou E (2017) Augmented reality application to support remote maintenance as a service in the robotics Industry. Procedia CIRP 63:46-51 [37] Hořejší P (2015) Augmented reality system for virtual training of parts assembly. Procedia Eng 100:699-706 [38] Füchter SK, Schlichting MS, Salazar G (2021) Aeronautic pilot training and augmented reality. Acta Imeko 10(3):66-71 [39] Fründ J, Gausemeier J, Matysczok C et al (2005) Using augmented reality technology to support the automobile development. In: computer supported cooperative work in design I: 8th international conference, CSCWD 2004, Xiamen, 26-28 May, Xiamen, China. [40] Ceruti A, Marzocca P, Liverani A et al (2019) Maintenance in aeronautics in an Industry 4.0 context: the role of augmented reality and additive manufacturing. J Comput Des Eng 6(4):516-526 [41] Mourtzis D, Zogopoulos V, Xanthi F (2019) Augmented reality application to support the assembly of highly customized products and to adapt to production re-scheduling. Int J Adv Manuf Technol 105(9):3899-3910 [42] Frigo MA, da Silva E, Barbosa GF (2016) Augmented reality in aerospace manufacturing: a review. J Ind Intell Inf 4(2):66. https://doi.org/10.18178/jiii.4.2.125-130 [43] Jeffri NFS, Rambli DRA (2021) A review of augmented reality systems and their effects on mental workload and task performance. Heliyon 7(3):06277. https://doi.org/10.1016/j.heliyon.2021.e06277 [44] de Oliveira ME, Corrêa CG (2020) Virtual reality and augmented reality applications in agriculture: a literature review. In: 2020 22nd symposium on virtual and augmented reality (SVR). IEEE, pp 1-9 [45] Woll R, Damerau T, Wrasse K et al (2011) Augmented reality in a serious game for manual assembly processes. In: 2011 IEEE international symposium on mixed and augmented reality-arts, media, and humanities. IEEE, pp 37-39 [46] Barfield W, Caudell T (2001) Boeing’s wire bundle assembly project. In: fundamentals of wearable computers and augmented reality. CRC Press, pp 462-482 [47] Liu Y, Li S, Wang J et al (2015) A computer vision-based assistant system for the assembly of narrow cabin products. Int J Adv Manuf Technol 76(1):281-293 [48] Lai ZH, Tao W, Leu MC et al (2020) Smart augmented reality instructional system for mechanical assembly towards worker-centered intelligent manufacturing. J Manuf Syst 55:69-81 [49] Richardson T, Gilbert S, Holub J et al (2014) Fusing self-reported and sensor data from mixed-reality training [50] Garrido-Jurado S, Muñoz-Salinas R, Madrid-Cuevas FJ et al (2016) Generation of fiducial marker dictionaries using mixed integer linear programming, pattern recognition. 51:481-491. https://doi.org/10.1016/j.patcog.2015.09.023 [51] Evans G, Miller J, Pena MI et al (2017) Evaluating the microsoft hololens through an augmented reality assembly application. In: degraded environments: sensing, processing, and display 2017, vol 10197. SPIE, pp 282-297 [52] Ariansyah D, Erkoyuncu JA, Eimontaite I et al (2022) A head mounted augmented reality design practice for maintenance assembly: toward meeting perceptual and cognitive needs of ar users. Appl Ergon 98:103597. https://doi.org/10.1016/j.apergo.2021.103597 [53] Vorraber W, Gasser J, Webb H et al (2020) Assessing augmented reality in production: remote-assisted maintenance with hololens. Procedia CIRP 88:139-144 [54] Brunzini A, Mandolini M, Caragiuli M, Germani M et al (2021) Hololens 2 for maxillofacial surgery: a preliminary study. In: international conference on design, simulation, manufacturing: the innovation exchange. Springer, pp 133-140 [55] Leu MC, ElMaraghy HA, Nee AY et al (2013) Cad model based virtual assembly simulation, planning and training. CIRP Ann 62(2):799-822 [56] Webel S, Becker M, Stricker D et al (2007) Identifying differences between cad and physical mock-ups using ar. In: 2007 6th IEEE and ACM international symposium on mixed and augmented reality. IEEE, pp 281-282 [57] Azpiazu J, Siltanen S , Multanen P et al (2011) Remote support for maintenance tasks by the use of augmented reality: the ManuVAR project. CARVI 2011: IX congress on virtualreality applications, 11-12 November, Alava, Spain [58] Kwon HJ, Lee SI, Park JH et al (2021) Design of augmented reality train-ing content for railway vehicle maintenance focusing on the axle-mounted disc brake system. Appl Sci 11(19):9090. https://doi.org/10.3390/app11199090 [59] Yi B, Sun R, Long L et al (2022) From coarse to fine: an augmented reality-based dynamic inspection method for visualized railway routing of freight cars. Meas Sci Technol 33(5):055013. https://doi.org/10.1088/1361-6501/ac3c1c [60] Seeliger A, Netland T, Feuerriegel S (2022) Aug-mented reality for machine setups: task performance and usability evaluation in a field test. Procedia CIRP 107:570-575 [61] Teruggi S, Fassi F (2022) Hololens 2 spatial map-ping capabilities in vast monumental heritage environments. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLVI-2/W1-2022, pp. 489-496 [62] Unger M, Heinrich S, Rick M et al (2022) Hologram accuracy evaluation of hololens 2 for thermal imaging in medical applications. Curr Direct Biomed Eng 8(2):193-196 [63] Curless B, Levoy M (1996) A volumetric method for building complex models from range images. In: proceedings of the 23rd annual conference on computer graphics and interactive techniques, pp 303-312 [64] Marino E, Barbieri L, Colacino B et al (2021) An augmented reality inspection tool to support workers in Industry 4.0 environments. Comput Ind 127:103412. https://doi.org/10.1016/j.compind.2021.103412 [65] Lu Y, Zheng H, Chand S et al (2022) Outlook on human-centric manufacturing towards Industry 5.0. J Manuf Syst 62:612-627 [66] Cao A, Chintamani KK, Pandya AK et al (2009) Nasa tlx: software for assessing subjective mental workload. Behav Res Methods 41(1):113-117 [67] Loizeau Q, Danglade F, Ababsa F et al (2019) Evaluating added value of augmented reality to assist aeronautical maintenance workers—experimentation on on-field use case. In: international conference on virtual reality and augmented reality. Springer, pp 151-169 [68] Re GM, Oliver J, Bordegoni M (2016) Impact of monitor-based augmented reality for on-site industrial manual operations. Cognit Technol Work 18(2):379-392 [69] Jordan PW, Thomas B, McClelland IL et al (1996) Usability evaluation in industry, CRC Press, Boca Raton [70] Fischini A, Ababsa F, Grasser M (2018) Usability of augmented reality in aeronautic maintenance, repair and overhaul. International conference on artificial reality and telexistence and Eurographics symposium on virtual environments, Nov 2018, Limassol, Cyprus. |
| [1] | Dien Hu, Jieyichen Fang, Feng Zeng, Ming-Wang Fu. Grain size effect on the assembly quality of micro-scaled barrel formed by microforming[J]. Advances in Manufacturing, 2024, 12(1): 19-32. |
| [2] | Zhi-Yong Zhang, Xiao-Dong Wang, Tong-Qun Ren, Tian-Lun Jin. Novel gripper module and method for automated assembly of miniature parts[J]. Advances in Manufacturing, 2023, 11(2): 295-310. |
| [3] | Li-Ping Zhao, Bo-Hao Li, Yi-Yong Yao. A novel predict-prevention quality control method of multi-stage manufacturing process towards zero defect manufacturing[J]. Advances in Manufacturing, 2023, 11(2): 280-294. |
| [4] | Qi Zhong, Xiao-Yong Tian, Xiao-Kang Huang, Zhi-Qiang Tong, Yi Cao, Di-Chen Li. High-accuracy calibration for multi-laser powder bed fusion via in situ detection and parameter identification[J]. Advances in Manufacturing, 2022, 10(4): 556-570. |
| [5] | Y. K. Liu, S. K. Ong, A. Y. C. Nee. State-of-the-art survey on digital twin implementations[J]. Advances in Manufacturing, 2022, 10(1): 1-23. |
| [6] | Jian-Hua Liu, Xia-Yu Li, Huan-Xiong Xia, Lei Guo. Effects of assembly errors and bonding defects on the centroid drift of a precision sleeve structure[J]. Advances in Manufacturing, 2021, 9(4): 509-519. |
| [7] | C. Y. Siew, S. K. Ong, A. Y. C. Nee. Improving maintenance effi ciency and safety through a human-centric approach[J]. Advances in Manufacturing, 2021, 9(1): 104-114. |
| [8] | S. K. Ong, X. Wang, A. Y. C. Nee. 3D bare-hand interactions enabling ubiquitous interactions with smart objects[J]. Advances in Manufacturing, 2020, 8(2): 133-143. |
| [9] | S. K. Ong, A. Y. C. Nee, A. W. W. Yew, N. K. Thanigaivel. AR-assisted robot welding programming[J]. Advances in Manufacturing, 2020, 8(1): 40-48. |
| [10] | Jia-Hua Zhang, Ai-Ping Li, Xue-Mei Liu. Hybrid genetic algorithm for a type-II robust mixed-model assembly line balancing problem with interval task times[J]. Advances in Manufacturing, 2019, 7(2): 117-132. |
| [11] | D. Ni, A. W. W. Yew, S. K. Ong, A. Y. C. Nee. Haptic and visual augmented reality interface for programming welding robots[J]. Advances in Manufacturing, 2017, 5(3): 191-198. |
| [12] | X. Wang, S. K. Ong, A. Y. C. Nee. A comprehensive survey of augmented reality assembly research[J]. Advances in Manufacturing, 2016, 4(1): 1-22. |
| [13] | H. C. Fang, S. K. Ong, A. Y. C. Nee. Novel AR-based interface for human-robot interaction and visualization[J]. Advances in Manufacturing, 2014, 2(4): 275-288. |
| [14] | K. E. Aasmundtveit B. Q. Ta Q.-H. Nguyen T. B. Haugen N. Hoivik E. Halv. Local synthesis of carbon nanotubes for direct integration in Simicrosystems – design considerations[J]. Advances in Manufacturing, 2013, 1(3): 218-225. |
| [15] | Feng-Feng Xi, Lin Yu,Xiao-Wei Tu. Framework on robotic percussive riveting for aircraft assembly automation[J]. Advances in Manufacturing, 2013, 1(1): 112-122. |
| 阅读次数 | ||||||
|
全文 |
|
|||||
|
摘要 |
|
|||||
Tel: 86-21-66135510
Fax: 86-21-66132736
E-mail: aim@oa.shu.edu.cn
