Intuitive and efficient interfaces for humanrobot
interaction (HRI) has been a challenging issue in
robotics as it is essential for the prevalence of robots
supporting humans in key areas of activities. This paper
presents a novel augmented reality (AR) based interface to
facilitate human-virtual robot interaction. A number of
human-virtual robot interaction methods have been formulated
and implemented with respect to the various types
of operations needed in different robotic applications.
A Euclidean distance-based method is developed to assist
the users in the interaction with the virtual robot and the
spatial entities in an AR environment. A monitor-based
visualization mode is adopted as it enables the users to
perceive the virtual contents associated with different
interaction methods, and the virtual content augmented in
the real environment is informative and useful to the users
during their interaction with the virtual robot. Case
researches are presented to demonstrate the successful
implementation of the AR-based HRI interface in planning
robot pick-and-place operations and path following
operations.
H. C. Fang
,
S. K. Ong
,
A. Y. C. Nee
. Novel AR-based interface for human-robot interaction
and visualization[J]. Advances in Manufacturing, 2014
, 2(4)
: 275
-288
.
DOI: 10.1007/s40436-014-0087-9
1. Meeussen W (2006) Compliant robot motion: from path planning or human demonstration to force controlled task execution. Dissertation,Katholieke Universiteit Leuven
2. Burdea GC (1996) Virtual reality and robotics in medicine. In:IEEE International Workshop on Robot and Human Communication,Tsukuba, 11–14 Nov 1996, pp 16–25
3. Castillo CRA, Wahrburg J (2007) Improving robot arm control for safe and robust haptic cooperation in orthopaedic procedures. Int J Med Robot Comput Assist Surg 3(4):316–322
4. Breazeal C, Edsinger A, Fitzpatrick P et al (2001) Active vision for sociable robots. IEEE Trans Syst Man Cybern A
31(5):443–453
5. Rastogi A, Milgram P, Drascic D et al (1996) Tele-robotic control with stereoscopic augmented reality. In: Proceedings of the SPIE stereoscopic displays and virtual reality systems III, San Jose, CA, 30 Jan–2 Feb 1996, vol 2653. doi:10.1117/12.237424
6. Xiong Y, Li S, Xie M (2006) Predictive display and interaction of tele-robots based on augmented reality. Robotica 24(4):447–453
7. Green SA, Chase JG, Chen XQ et al (2010) Evaluating the augmented reality human-robot collaboration system. Int J Intell Syst Technol Appl 8(1/4):130–143
8. Zaeh MF, Vogl W (2006) Interactive laser-projection for programming industrial robots. In: Proceedings of the international symposium on mixed and augmented reality, Santa Barbara,22–25 Oct 2006, pp 125–133
9. Reinhart G, Munzert U, Vogl W (2008) A programming system for robot-based remote-laser-welding with conventional optics.CIRP Ann 57(1):37–40
10. Collett THJ, MacDonald B (2006) Developer oriented visualization of a robot program. In: Proceedings of the conference on human–robot interaction. Salt Lake City, 2–4 Mar 2006,pp 49–55
11. Bischoff R, Kazi A, Seyfarth M (2002) The MORPHA style guide for icon-based programming. In: Proceedings of the IEEE international workshop on robot and human interactive communication.Berlin, 25–27 Sep 2002, pp 482–491
12. Chintamani K, Cao A, Ellis RD et al (2010) Improved telemanipulator navigation during display-control misalignments using augmented reality cues. IEEE Trans Syst Man Cybern A 40(1):29–39
13. Stilman M, Michel P, Chestnutt J et al (2005) Augmented reality for robot development and experimentation. Tech Rep CMU-RITR-05-55. Robotics Institute, Carnegie Mellon University,Pittsburgh
14. Mohareri O, Rad AB (2011) Autonomous humanoid robot navigation using augmented reality technique. In: Proceedings of the IEEE international conference on mechatronics, Istanbul, 13–15 April 2011, pp 463–471
15. Parasuraman R, Sheridan TB, Wickens CD (2000) A model for types and levels of human interaction with automation. IEEE Trans Syst Man Cybern A 30(3):286–297
16. Thrun S (2004) Toward a framework for human-robot interaction. Hum-Comput Interact 19(1/2):9–24
17. Goodrich MA, Schultz AC (2007) Human-robot interaction: a survey. Found Trends Hum-Comput Interact 1(3):203–275
18. Prusak A, Bemshausen J, Roth H et al (2007) Applications of automated guided vehicle (AGV) and industry robots with PMDcamera.In: Proceedings of the international conference on robotics and applications, Wurzburg, 29–31 Aug 2007,pp 299–303
19. Yanagihara Y, Kakizaki T, Arakawa K et al (1998) A multimodal teaching advisor for sensor-enhanced robotic systems in manufacturing.Robot Comput-Integr Manuf 14(4):263–273
20. Kock S, Bredahl J, Eriksson PJ et al (2006) Taming the robot—better safety without higher fences. ABB Rev 4:11–15
21. Adams JA (2002) Critical considerations for human-robot interface development. In: Proceedings of 2002 AAAI fall symposium,North Falmouth, 15–17 November 2002. Tech. Report FS-02-03, pp 1–8
22. Steinfeld A, Fong T, Kaber D et al (2006) Common metrics for human–robot interaction. In: Proceeding of the ACM international conference on human-robot interaction. Salt Lake City, 2–3 Mar 2006, pp 33–40
23. Endsley MR (2000) Theoretical underpinnings of situation awareness: a critical review. In: Endsley MR, Garland DJ (eds) Situation awareness analysis and measurement. Lawrence Erlbaum Associates, Mahwah, pp 3–32
24. Endsley MR (2000) Direct measurement of situation awareness:validity and use of SAGAT. In: Endsley MR, Garland DJ (eds) Situation awareness analysis and measurement. Lawrence Erlbaum Associates, Mahwah, pp 147–173
25. Murphy RR (2004) Human-robot interaction in rescue robotics. IEEE Trans Syst Man Cybern C 34(2):138–153
26. Heyer C (2010) Human-robot interaction and future industrial robotics applications. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems, Taipei 18–22 Oct 2010, pp 4749–4754
27. Thrun S, Bennewitz M, Burgard W et al (1999) MINERVA: a second generation mobile tour-guide robot. In: Proceedings of the IEEE international conference on robotics and automation,etroit, 10–15 May 1999, pp 1999–2005
28. Waldherr S, Romero R, Thrun S (2000) A gesture based interface for human-robot interaction. Auton Robots 9(2):151–173
29. Yin X, Zhu X (2006) Hand posture recognition in gesture-based human-robot interaction. In: Proceedings of the IEEE international conference on industrial electronics and applications. Singapore, 24–26 May 2006, pp 1–6
30. Tang X, Yamada H, Zhao D et al (2009) Haptic interaction in teleoperation control system of construction robot based on virtual reality. In: Proceedings of the IEEE international conference on mechatronics and automation, Changchun. 9–12 Aug 2009,pp 78–83
31. Tsetserukou D, Sato K, Kawakami N et al (2009) Teleoperation system with haptic feedback for physical interaction with remote environment. In: Proceedings of the ICROS-SICE international joint conference, Fukuoka 18–21 Aug 2009, pp 3353–3358
32. Pires JN (2005) Robot-by-voice: experiments on commanding an industrial robot using the human voice. Industrial Robot 32(6):505–511
33. Medicherla H, Sekmen A (2007) Human-robot interaction via voice-controllable intelligent user interface. Robotica 25(5):521–527
34. Iba S, Paredis CJJ, Khosla PK (2005) Interactive multimodal robot Programming. Int J Robot Res 24(1):83–104
35. Mar?′n R, Sanz PJ, Nebot P et al (2005) A multimodal interface to control a robot arm via the web: a case study on remote programming.IEEE Trans Ind Electron 52(6):1506–1520
36. Pires JN, Godinho T, Araujo R (2007) Using digital pens to program welding tasks. Ind Robot 34(6):476–486
37. Schraft RD, Meyer C (2006) The need for an intuitive teaching method for small and medium enterprises. VDI Ber 1956:95
38. Chen J, Zelinsky A (2003) Programming by demonstration: coping with suboptimal teaching actions. Int J Robot Res
22(5):299–319
39. Aleotti J, Caselli S (2005) Trajectory clustering and stochastic approximation for robot programming by demonstration. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems, Edmonton, 2–6 Aug 2005,pp 1029–1034
40. Khatib O, Sentis L, Park JH et al (2004) Whole body dynamic behavior and control of human-like robots. Int J Humanoid Robot 1(1):29–43
41. Z ˇ lajpah L (2008) Simulation in robotics. Math Comput Simul 79(4):879–897
42. Ehrenmann M, Ambela D, Steinaus P et al (2000) A comparison of four fast vision based object recognition methods for programming by demonstration applications. In: Proceedings of the IEEE international conference on robotics and automation, San Francisco 24–28 April 2000, pp 1862–1867
43. Chong JWS, Ong SK, Nee AYC et al (2009) Robot programming using augmented reality: an interactive method for planning collision-free paths. Robot Comput-Integr Manuf 25(3):689–701
44. Fang HC, Ong SK, Nee AYC (2012) Interactive robot trajectory planning and simulation using augmented reality. Robot Comput-Integr Manuf 28(2):227–237
45. Han BO, Kim YH, Cho K et al (2010) Museum tour guide robot with augmented reality. In: Proceedings of the international conference on virtual systems and multimedia, Seoul, 20–23 Oct 2010, pp 223–229
46. Ong SK, Chong JWS, Nee AYC (2010) A novel AR-based robot programming and path planning methodology. Robot Comput-Integr Manuf 26(3):240–249
