Download a PDF with the full list of our publications: Robot-Intelligence-Lab-Publications-2021.pdf
A comprehensive list can also be found at Google Scholar, or by searching for the publications of author Kormushev, Petar.
Results
- Showing results for:
- Reset all filters
Search results
-
Journal articleAlAttar A, Kormushev P, 2020,
Kinematic-model-free orientation control for robot manipulation using locally weighted dual quaternions
, Robotics, Vol: 9, Pages: 1-12, ISSN: 2218-6581Conventional control of robotic manipulators requires prior knowledge of their kinematic structure. Model-learning controllers have the advantage of being able to control robots without requiring a complete kinematic model and work well in less structured environments. Our recently proposed Encoderless controller has shown promising ability to control a manipulator without requiring any prior kinematic model whatsoever. However, this controller is only limited to position control, leaving orientation control unsolved. The research presented in this paper extends the state-of-the-art kinematic-model-free controller to handle orientation control to manipulate a robotic arm without requiring any prior model of the robot or any joint angle information during control. This paper presents a novel method to simultaneously control the position and orientation of a robot’s end effector using locally weighted dual quaternions. The proposed novel controller is also scaled up to control three-degrees-of-freedom robots.
-
Journal articleCursi F, Mylonas GP, Kormushev P, 2020,
Adaptive kinematic modelling for multiobjective control of a redundant surgical robotic tool
, Robotics, Vol: 9, Pages: 68-68, ISSN: 2218-6581Accurate kinematic models are essential for effective control of surgical robots. For tendon driven robots, which are common for minimally invasive surgery, the high nonlinearities in the transmission make modelling complex. Machine learning techniques are a preferred approach to tackle this problem. However, surgical environments are rarely structured, due to organs being very soft and deformable, and unpredictable, for instance, because of fluids in the system, wear and break of the tendons that lead to changes of the system’s behaviour. Therefore, the model needs to quickly adapt. In this work, we propose a method to learn the kinematic model of a redundant surgical robot and control it to perform surgical tasks both autonomously and in teleoperation. The approach employs Feedforward Artificial Neural Networks (ANN) for building the kinematic model of the robot offline, and an online adaptive strategy in order to allow the system to conform to the changing environment. To prove the capabilities of the method, a comparison with a simple feedback controller for autonomous tracking is carried out. Simulation results show that the proposed method is capable of achieving very small tracking errors, even when unpredicted changes in the system occur, such as broken joints. The method proved effective also in guaranteeing accurate tracking in teleoperation.
-
Journal articleFalck F, Doshi S, Tormento M, et al., 2020,
Robot DE NIRO: a human-centered, autonomous, mobile research platform for cognitively-enhanced manipulation
, Frontiers in Robotics and AI, Vol: A17, ISSN: 2296-9144We introduceRobot DE NIRO, an autonomous, collaborative, humanoid robot for mobilemanipulation. We built DE NIRO to perform a wide variety of manipulation behaviors, with afocus on pick-and-place tasks. DE NIRO is designed to be used in a domestic environment,especially in support of caregivers working with the elderly. Given this design focus, DE NIRO caninteract naturally, reliably, and safely with humans, autonomously navigate through environmentson command, intelligently retrieve or move target objects, and avoid collisions efficiently. Wedescribe DE NIRO’s hardware and software, including an extensive vision sensor suite of 2Dand 3D LIDARs, a depth camera, and a 360-degree camera rig; two types of custom grippers;and a custom-built exoskeleton called DE VITO. We demonstrate DE NIRO’s manipulationcapabilities in three illustrative challenges: First, we have DE NIRO perform a fetch-an-objectchallenge. Next, we add more cognition to DE NIRO’s object recognition and grasping abilities,confronting it with small objects of unknown shape. Finally, we extend DE NIRO’s capabilitiesinto dual-arm manipulation of larger objects. We put particular emphasis on the features thatenable DE NIRO to interact safely and naturally with humans. Our contribution is in sharinghow a humanoid robot with complex capabilities can be designed and built quickly with off-the-shelf hardware and open-source software. Supplementary material including our code, adocumentation, videos and the CAD models of several hardware parts are openly availableavailable athttps://www.imperial.ac.uk/robot-intelligence/software/
-
Conference paperPardo F, Levdik V, Kormushev P, 2020,
Scaling all-goals updates in reinforcement learning using convolutional neural networks
, 34th AAAI Conference on Artificial Intelligence (AAAI 2020), Publisher: Association for the Advancement of Artificial Intelligence, Pages: 5355-5362, ISSN: 2374-3468Being able to reach any desired location in the environmentcan be a valuable asset for an agent. Learning a policy to nav-igate between all pairs of states individually is often not fea-sible. Anall-goals updatingalgorithm uses each transitionto learn Q-values towards all goals simultaneously and off-policy. However the expensive numerous updates in parallellimited the approach to small tabular cases so far. To tacklethis problem we propose to use convolutional network archi-tectures to generate Q-values and updates for a large numberof goals at once. We demonstrate the accuracy and generaliza-tion qualities of the proposed method on randomly generatedmazes and Sokoban puzzles. In the case of on-screen goalcoordinates the resulting mapping from frames todistance-mapsdirectly informs the agent about which places are reach-able and in how many steps. As an example of applicationwe show that replacing the random actions inε-greedy ex-ploration by several actions towards feasible goals generatesbetter exploratory trajectories on Montezuma’s Revenge andSuper Mario All-Stars games.
-
Conference paperSaputra RP, Rakicevic N, Kormushev P, 2020,
Sim-to-real learning for casualty detection from ground projected point cloud data
, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019), Publisher: IEEEThis paper addresses the problem of human body detection-particularly a human body lying on the ground (a.k.a. casualty)-using point cloud data. This ability to detect a casualty is one of the most important features of mobile rescue robots, in order for them to be able to operate autonomously. We propose a deep-learning-based casualty detection method using a deep convolutional neural network (CNN). This network is trained to be able to detect a casualty using a point-cloud data input. In the method we propose, the point cloud input is pre-processed to generate a depth image-like ground-projected heightmap. This heightmap is generated based on the projected distance of each point onto the detected ground plane within the point cloud data. The generated heightmap-in image form-is then used as an input for the CNN to detect a human body lying on the ground. To train the neural network, we propose a novel sim-to-real approach, in which the network model is trained using synthetic data obtained in simulation and then tested on real sensor data. To make the model transferable to real data implementations, during the training we adopt specific data augmentation strategies with the synthetic training data. The experimental results show that data augmentation introduced during the training process is essential for improving the performance of the trained model on real data. More specifically, the results demonstrate that the data augmentations on raw point-cloud data have contributed to a considerable improvement of the trained model performance.
-
Journal articleRakicevic N, Kormushev P, 2019,
Active learning via informed search in movement parameter space for efficient robot task learning and transfer
, Autonomous Robots, Vol: 43, Pages: 1917-1935, ISSN: 0929-5593Learning complex physical tasks via trial-and-error is still challenging for high-degree-of-freedom robots. Greatest challenges are devising a suitable objective function that defines the task, and the high sample complexity of learning the task. We propose a novel active learning framework, consisting of decoupled task model and exploration components, which does not require an objective function. The task model is specific to a task and maps the parameter space, defining a trial, to the trial outcome space. The exploration component enables efficient search in the trial-parameter space to generate the subsequent most informative trials, by simultaneously exploiting all the information gained from previous trials and reducing the task model’s overall uncertainty. We analyse the performance of our framework in a simulation environment and further validate it on a challenging bimanual-robot puck-passing task. Results show that the robot successfully acquires the necessary skills after only 100 trials without any prior information about the task or target positions. Decoupling the framework’s components also enables efficient skill transfer to new environments which is validated experimentally.
-
Conference paperFalck F, Doshi S, Smuts N, et al., 2019,
Human-centered manipulation and navigation with robot DE NIRO
Social assistance robots in health and elderly care have the potential tosupport and ease human lives. Given the macrosocial trends of aging andlong-lived populations, robotics-based care research mainly focused on helpingthe elderly live independently. In this paper, we introduce Robot DE NIRO, aresearch platform that aims to support the supporter (the caregiver) and alsooffers direct human-robot interaction for the care recipient. Augmented byseveral sensors, DE NIRO is capable of complex manipulation tasks. It reliablyinteracts with humans and can autonomously and swiftly navigate throughdynamically changing environments. We describe preliminary experiments in ademonstrative scenario and discuss DE NIRO's design and capabilities. We putparticular emphases on safe, human-centered interaction procedures implementedin both hardware and software, including collision avoidance in manipulationand navigation as well as an intuitive perception stack through speech and facerecognition.
-
Conference paperTavakoli A, Levdik V, Islam R, et al., 2019,
Exploring Restart Distributions
, Montréal, Canada, The Fourth Multidisciplinary Conference on Reinforcement Learning and Decision Making, Publisher: arXivWe consider the generic approach of using an experience memory to help exploration by adapting a restart distribution. That is, given the capacity to reset the state with those corresponding to the agent's past observations, we help exploration by promoting faster state-space coverage via restarting the agent from a more diverse set of initial states, as well as allowing it to restart in states associated with significant past experiences. This approach is compatible with both on-policy and off-policy methods. However, a caveat is that altering the distribution of initial states could change the optimal policies when searching within a restricted class of policies. To reduce this unsought learning bias, we evaluate our approach in deep reinforcement learning which benefits from the high representational capacity of deep neural networks. We instantiate three variants of our approach, each inspired by an idea in the context of experience replay. Using these variants, we show that performance gains can be achieved, especially in hard exploration problems.
-
Conference paperFalck F, Larppichet K, Kormushev P, 2019,
DE VITO: A dual-arm, high degree-of-freedom, lightweight, inexpensive, passive upper-limb exoskeleton for robot teleoperation
, TAROS: Annual Conference Towards Autonomous Robotic Systems, Publisher: Springer, ISSN: 0302-9743While robotics has made significant advances in perception, planning and control in recent decades, the vast majority of tasks easily completed by a human, especially acting in dynamic, unstructured environments, are far from being autonomously performed by a robot. Teleoperation, remotely controlling a slave robot by a human operator, can be a realistic, complementary transition solution that uses the motion intelligence of a human in complex tasks while exploiting the robot’s autonomous reliability and precision in less challenging situations.We introduce DE VITO, a seven degree-of-freedom, dual-arm upper-limb exoskeleton that passively measures the pose of a human arm. DE VITO is a lightweight, simplistic and energy-efficient design with a total material cost of at least an order of magnitude less than previous work. Given the estimated human pose, we implement both joint and Cartesian space kinematic control algorithms and present qualitative experimental results on various complex manipulation tasks teleoperating Robot DE NIRO, a research platform for mobile manipulation, that demonstrate the functionality of DE VITO. We provide the CAD models, open-source code and supplementary videos of DE VITO at http://www.imperial.ac.uk/robot-intelligence/robots/de_vito/.
-
Conference paperAlAttar A, Rouillard L, Kormushev P, 2019,
Autonomous air-hockey playing cobot using optimal control and vision-based Bayesian tracking
, Towards Autonomous Robotic Systems, Publisher: Springer, ISSN: 0302-9743This paper presents a novel autonomous air-hockey playing collaborative robot (cobot) that provides human-like gameplay against human opponents. Vision-based Bayesian tracking of the puck and striker are used in an Analytic Hierarchy Process (AHP)-based probabilistic tactical layer for high-speed perception. The tactical layer provides commands for an active control layer that controls the Cartesian position and yaw angle of a custom end effector. The active layer uses optimal control of the cobot’s posture inside the task nullspace. The kinematic redundancy is resolved using a weighted Moore-Penrose pseudo-inversion technique. Experiments with human players show high-speed human-like gameplay with potential applications in the growing field of entertainment robotics.
-
Journal articleKormushev P, Ugurlu B, Caldwell DG, et al., 2019,
Learning to exploit passive compliance for energy-efficient gait generation on a compliant humanoid
, Autonomous Robots, Vol: 43, Pages: 79-95, ISSN: 0929-5593Modern humanoid robots include not only active compliance but also passive compliance. Apart from improved safety and dependability, availability of passive elements, such as springs, opens up new possibilities for improving the energy efficiency. With this in mind, this paper addresses the challenging open problem of exploiting the passive compliance for the purpose of energy efficient humanoid walking. To this end, we develop a method comprising two parts: an optimization part that finds an optimal vertical center-of-mass trajectory, and a walking pattern generator part that uses this trajectory to produce a dynamically-balanced gait. For the optimization part, we propose a reinforcement learning approach that dynamically evolves the policy parametrization during the learning process. By gradually increasing the representational power of the policy parametrization, it manages to find better policies in a faster and computationally efficient way. For the walking generator part, we develop a variable-center-of-mass-height ZMP-based bipedal walking pattern generator. The method is tested in real-world experiments with the bipedal robot COMAN and achieves a significant 18% reduction in the electric energy consumption by learning to efficiently use the passive compliance of the robot.
-
Conference paperWang K, Shah A, Kormushev P, 2018,
SLIDER: A Bipedal Robot with Knee-less Legs and Vertical Hip Sliding Motion
, 21st International Conference on Climbing and Walking Robots and Support Technologies for Mobile Machines (CLAWAR 2018) -
Conference paperPardo F, Levdik V, Kormushev P, 2018,
Q-map: A convolutional approach for goal-oriented reinforcement learning.
Goal-oriented learning has become a core concept in reinforcement learning(RL), extending the reward signal as a sole way to define tasks. However, asparameterizing value functions with goals increases the learning complexity,efficiently reusing past experience to update estimates towards several goalsat once becomes desirable but usually requires independent updates per goal.Considering that a significant number of RL environments can support spatialcoordinates as goals, such as on-screen location of the character in ATARI orSNES games, we propose a novel goal-oriented agent called Q-map that utilizesan autoencoder-like neural network to predict the minimum number of stepstowards each coordinate in a single forward pass. This architecture is similarto Horde with parameter sharing and allows the agent to discover correlationsbetween visual patterns and navigation. For example learning how to use aladder in a game could be transferred to other ladders later. We show how thisnetwork can be efficiently trained with a 3D variant of Q-learning to updatethe estimates towards all goals at once. While the Q-map agent could be usedfor a wide range of applications, we propose a novel exploration mechanism inplace of epsilon-greedy that relies on goal selection at a desired distancefollowed by several steps taken towards it, allowing long and coherentexploratory steps in the environment. We demonstrate the accuracy andgeneralization qualities of the Q-map agent on a grid-world environment andthen demonstrate the efficiency of the proposed exploration mechanism on thenotoriously difficult Montezuma's Revenge and Super Mario All-Stars games.
-
Conference paperSaputra RP, Kormushev P, 2018,
Casualty detection from 3D point cloud data for autonomous ground mobile rescue robots
, SSRR 2018, Publisher: IEEEOne of the most important features of mobilerescue robots is the ability to autonomously detect casualties,i.e. human bodies, which are usually lying on the ground. Thispaper proposes a novel method for autonomously detectingcasualties lying on the ground using obtained 3D point-clouddata from an on-board sensor, such as an RGB-D camera ora 3D LIDAR, on a mobile rescue robot. In this method, theobtained 3D point-cloud data is projected onto the detectedground plane, i.e. floor, within the point cloud. Then, thisprojected point cloud is converted into a grid-map that isused afterwards as an input for the algorithm to detecthuman body shapes. The proposed method is evaluated byperforming detections of a human dummy, placed in differentrandom positions and orientations, using an on-board RGB-Dcamera on a mobile rescue robot called ResQbot. To evaluatethe robustness of the casualty detection method to differentcamera angles, the orientation of the camera is set to differentangles. The experimental results show that using the point-clouddata from the on-board RGB-D camera, the proposed methodsuccessfully detects the casualty in all tested body positions andorientations relative to the on-board camera, as well as in alltested camera angles.
-
Conference paperSaputra RP, Kormushev P, 2018,
Casualty detection for mobile rescue robots via ground-projected point clouds
, Towards Autonomous Robotic Systems (TAROS) 2018, Publisher: Springer, Cham, Pages: 473-475, ISSN: 0302-9743In order to operate autonomously, mobile rescue robots needto be able to detect human casualties in disaster situations. In this paper,we propose a novel method for autonomous detection of casualties lyingdown on the ground based on point-cloud data. This data can be obtainedfrom different sensors, such as an RGB-D camera or a 3D LIDAR sensor.The method is based on a ground-projected point-cloud (GPPC) imageto achieve human body shape detection. A preliminary experiment hasbeen conducted using the RANSAC method for floor detection and, theHOG feature and the SVM classifier to detect human body shape. Theresults show that the proposed method succeeds to identify a casualtyfrom point-cloud data in a wide range of viewing angles.
-
Conference paperPardo F, Tavakoli A, Levdik V, et al., 2018,
Time limits in reinforcement learning
, International Conference on Machine Learning, Pages: 4042-4051In reinforcement learning, it is common to let anagent interact for a fixed amount of time with itsenvironment before resetting it and repeating theprocess in a series of episodes. The task that theagent has to learn can either be to maximize itsperformance over (i) that fixed period, or (ii) anindefinite period where time limits are only usedduring training to diversify experience. In thispaper, we provide a formal account for how timelimits could effectively be handled in each of thetwo cases and explain why not doing so can causestate-aliasing and invalidation of experience re-play, leading to suboptimal policies and traininginstability. In case (i), we argue that the termi-nations due to time limits are in fact part of theenvironment, and thus a notion of the remainingtime should be included as part of the agent’s in-put to avoid violation of the Markov property. Incase (ii), the time limits are not part of the envi-ronment and are only used to facilitate learning.We argue that this insight should be incorporatedby bootstrapping from the value of the state atthe end of each partial episode. For both cases,we illustrate empirically the significance of ourconsiderations in improving the performance andstability of existing reinforcement learning algo-rithms, showing state-of-the-art results on severalcontrol tasks.
-
Conference paperSaputra RP, Kormushev P, 2018,
ResQbot: a mobile rescue robot with immersive teleperception for casualty extraction
, Towards Autonomous Robotic Systems (TAROS) 2018, Publisher: Springer International Publishing AG, part of Springer Nature, Pages: 209-220, ISSN: 0302-9743In this work, we propose a novel mobile rescue robot equipped with an immersive stereoscopic teleperception and a teleoperation control. This robot is designed with the capability to perform safely a casualty-extraction procedure. We have built a proof-of-concept mobile rescue robot called ResQbot for the experimental platform. An approach called “loco-manipulation” is used to perform the casualty-extraction procedure using the platform. The performance of this robot is evaluated in terms of task accomplishment and safety by conducting a mock rescue experiment. We use a custom-made human-sized dummy that has been sensorised to be used as the casualty. In terms of safety, we observe several parameters during the experiment including impact force, acceleration, speed and displacement of the dummy’s head. We also compare the performance of the proposed immersive stereoscopic teleperception to conventional monocular teleperception. The results of the experiments show that the observed safety parameters are below key safety thresholds which could possibly lead to head or neck injuries. Moreover, the teleperception comparison results demonstrate an improvement in task-accomplishment performance when the operator is using the immersive teleperception.
-
Conference paperWang K, Shah A, Kormushev P, 2018,
SLIDER: a novel bipedal walking robot without knees
, Towards Autonomous Robotic Systems (TAROS) 2018, Publisher: Springer International Publishing AG, part of Springer Nature, Pages: 471-472, ISSN: 0302-9743In this work, we propose a novel mobile rescue robot equipped with an immersive stereoscopic teleperception and a teleoperation control. This robot is designed with the capability to perform safely a casualty-extraction procedure. We have built a proof-of-concept mobile rescue robot called ResQbot for the experimental platform. An approach called “loco-manipulation” is used to perform the casualty-extraction procedure using the platform. The performance of this robot is evaluated in terms of task accomplishment and safety by conducting a mock rescue experiment. We use a custom-made human-sized dummy that has been sensorised to be used as the casualty. In terms of safety, we observe several parameters during the experiment including impact force, acceleration, speed and displacement of the dummy’s head. We also compare the performance of the proposed immersive stereoscopic teleperception to conventional monocular teleperception. The results of the experiments show that the observed safety parameters are below key safety thresholds which could possibly lead to head or neck injuries. Moreover, the teleperception comparison results demonstrate an improvement in task-accomplishment performance when the operator is using the immersive teleperception.
-
Conference paperSaputra RP, Kormushev P, 2018,
ResQbot: A mobile rescue robot for casualty extraction
, 2018 ACM/IEEE International Conference on Human-Robot Interaction (HRI 2018), Publisher: Association for Computing Machinery, Pages: 239-240Performing search and rescue missions in disaster-struck environments is challenging. Despite the advances in the robotic search phase of the rescue missions, few works have been focused on the physical casualty extraction phase. In this work, we propose a mobile rescue robot that is capable of performing a safe casualty extraction routine. To perform this routine, this robot adopts a loco-manipulation approach. We have designed and built a mobile rescue robot platform called ResQbot as a proof of concept of the proposed system. We have conducted preliminary experiments using a sensorised human-sized dummy as a victim, to confirm that the platform is capable of performing a safe casualty extraction procedure.
-
Conference paperTavakoli A, Pardo F, Kormushev P, 2018,
Action branching architectures for deep reinforcement learning
, AAAI 2018, Publisher: AAAIDiscrete-action algorithms have been central to numerousrecent successes of deep reinforcement learning. However,applying these algorithms to high-dimensional action tasksrequires tackling the combinatorial increase of the numberof possible actions with the number of action dimensions.This problem is further exacerbated for continuous-actiontasks that require fine control of actions via discretization.In this paper, we propose a novel neural architecture fea-turing a shared decision module followed by several net-workbranches, one for each action dimension. This approachachieves a linear increase of the number of network outputswith the number of degrees of freedom by allowing a level ofindependence for each individual action dimension. To illus-trate the approach, we present a novel agent, called Branch-ing Dueling Q-Network (BDQ), as a branching variant ofthe Dueling Double Deep Q-Network (Dueling DDQN). Weevaluate the performance of our agent on a set of challeng-ing continuous control tasks. The empirical results show thatthe proposed agent scales gracefully to environments with in-creasing action dimensionality and indicate the significanceof the shared decision module in coordination of the dis-tributed action branches. Furthermore, we show that the pro-posed agent performs competitively against a state-of-the-art continuous control algorithm, Deep Deterministic PolicyGradient (DDPG).
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.
Contact us
