Artificial arms that assist my actions with little cognitive effort and can seamlessly be controlled concurrently and independently to my natural arms is a popular phantasm in science fiction. By allowing individuals to achieve otherwise impossible actions, such movement augmentation could revolutionise medical and industrial applications, and profoundly change the way humans interact with their environment. However, can the human brain deal with the complexity to control an extra limb and yield advantages from it? Our research addresses this question by i) designing interfaces and techniques to control supernumerary limbs, ii) carrying out psychophysical experiments to understand the major factors of movement augmentation, and iii) by developing applications.
With Carsten Mehring, we first analysed the neuromechanics and manipulation abilities of two subjects born with an extra finger on each hand (see left figure, Nature Communications 2019). Anatomical MRI of the supernumerary finger (SF) revealed that it is actuated by extra muscles and nerves, and fMRI identified a distinct cortical representation of the SF. Analysis of active exploration of objects and the results of a dedicated multi-fingers game showed that the two polydactyly subjects were able to coordinate the SF with their other fingers for more complex movements than five fingered subjects, giving them superior manipulation abilities without any apparent movement deficit elsewhere.
Elahe Abdi designed "trimanual" control studies with the hands and one foot movement controlling three virtual hands on a computer monitor to reach to objects. The results of a first study (PLoS ONE 2015) showed that the limbs moved simultaneously rather than serially without any specific difficulty to complete the tasks, and participants’ performance improved within a few minutes of practice. Subjective assessment indicated that controlling a third hand by foot required only negligible physical and mental efforts. A second study analysed the catching of three falling objects by three vs. two hands (Scientific Reports 2016). While the two strategies were similarly successful, three hand control was preferred and found easier, requiring less physical and mental load.
Yanpei Huang extended these experiments and developed an application for robot-assisted surgery. Yanpei first developed a passive haptic foot interface with four degrees-of-freedom and individual command patterns (IEEE T Mech 2020). She demonstrated that this interface enables faster and smoother control of a robot in accurate movements than an interface switching between different buttons to move in different directions (as is used to operate the endoscope in current commercial systems) (RAL 2019).
Using this interface, Yanpei then developed the trimanual control of a novel soft robotic endoscope equipped with two instruments (see right figure, Annals of Biomedical Engineering 2021). In an ex-vivo experiment, six subjects without surgical experience could successfully teleoperate the 13 degrees-of-freedom endoscope system with two instruments (with to cut the tissues at scattered target areas in a porcine stomach. Foot control yielded 43.7% faster task completion and required less mental effort as compared to a clutch-based hand control, which thus validating the three-limb tele-operation surgery concept.
Related publications
- C Mehring, M Akselrod, L Bashford, M Mace, H Choi, M Blueher, AS Buschhoff, T Pistohl, R Salomon, A Cheah, O Blanke, A Serino and E Burdet (2019), Augmented manipulation ability in humans with six-fingered hands. Nature Communications 10(1): 2401.
- J Eden, M Braecklein, J Ibanez Pereda, DY Barsakcioglu, G Di Pino, D Farina, E Burdet and C Mehring (2022), Principles of human movement augmentation and the challenges in making it a reality. Nature Communications 13(1): 1-13
- E Abdi, E Burdet, M Bouri and H Bleuler (2015), Control of a supernumerary robotic hand by foot: An experimental study in virtual reality. PLoS ONE 10(7): e0134501.
- E Abdi, E Burdet, M Bouri, S Himidan and H Bleuler (2016), In a demanding task, three-handed manipulation is preferred to two-handed manipulation. Scientific Reports 6: 21758.
- Y Huang, E Burdet, L Cao, PT Phan, AMH Tiong and SJ Phee (2020), A subject-specific four-degree-of- freedom foot interface to control a surgical robot. IEEE/ASME Transactions on Mechatronics 25 (2): 951-63.
- Y Huang, E Burdet, L Cao, PT Phan, AMH Tiong, P Zheng and SJ Phee (2019), Performance evaluation of a foot interface to operate a robot arm. IEEE Robotics and Automation Letters 4(4): 3302-9.
- Y Huang, J Eden, L Cao, E Burdet and SJ Phee (2020), Trimanipulation: An evaluation of human performance in 3-handed teleoperation. IEEE Transactions on Medical Robotics and Bionics 2(4): 545-8.
- Y Huang, W Lai, L Cao, SJ Phee and E Burdet (2020), Design and evaluation of a foot-controlled robotic endoscope system for endoscopic surgery. IEEE Robotics and Automation Letters 6(2): 2469-76.
- Y Huang, W Lai, L Cao, J Liu, E Burdet and SJ Phee (2020), A three-limb teleoperated robotic system with foot control for flexible endoscopic surgery. Annals of Biomedical Engineering 8: 1-5.
- A Noccaro, J Eden, G Di Pino, D Formica and E Burdet (2021), Human performance in three-hands tasks. Scientific Reports 11(1): 1-8.
- Y Huang, E Ivanova, J Eden, and E Burdet (2022), Identification of multiple limbs coordination strategies in a three-goal independent task. IEEE Transactions on Medical Robotics and Bionics 4(2): 348-51.
- M Braecklein, DY Barsakcioglu, J Ibanez Pereda, J Eden, E Burdet, C Mehring and D Farina (2022), The control and training of single motor units in isometric tasks are constrained by a common input signal. eLife 11: e72871.
- D Farina, E Burdet, C Mehring, and J Ibanez (2023), Roboticists want to give you a third arm: Unused bandwidth in neurons can be tapped to control extra limbs. IEEE Spectrum 60(3): 22-46.
- Y Huang, J Eden, E Ivanova and E Burdet (2023), Can training make three arms better than two heads for trimanual coordination? IEEE Open Journal of Engineering in Medicine and Biology 4: 148-55.