Cognitive Vision in Robotic Surgery

Search or filter publications

Search publications Search

Filter by type:

Filter by publication type

• Book
• Book chapter
• Conference paper
• Journal article
• Patent
• Report
• Software

Filter by year:

Filter from 20242023202220212020201920182017201620152014201320122011201020092008200720062005 to Filter to 20242023202220212020201920182017201620152014201320122011201020092008200720062005

---

Search results

• Conference paper
  Zhao L, Giannarou S, Lee S, Yang GZet al., 2016,

  Registration-free simultaneous catheter and environment modelling

  , Medical Image Computing and Computer Assisted Intervention (MICCAI) 2016, Publisher: Springer

  Endovascular procedures are challenging to perform due tothe complexity and difficulty in catheter manipulation. The simultaneousrecovery of the 3D structure of the vasculature and the catheter posi-tion and orientation intra-operatively is necessary in catheter controland navigation. State-of-art Simultaneous Catheter and EnvironmentModelling provides robust and real-time 3D vessel reconstruction based on real-time intravascular ultrasound (IVUS) imaging and electromagnetic (EM) sensing, but still relies on accurate registration between EM and pre-operative data. In this paper, a registration-free vessel reconstruction method is proposed for endovascular navigation. In the optimisation framework, the EM-CT registration is estimated and updated intra-operatively together with the 3D vessel reconstruction from IVUS, EM and pre-operative data, and thus does not require explicit registration. The proposed algorithm can also deal with global (patient) motion and periodic deformation caused by cardiac motion. Phantom and in-vivo experiments validate the accuracy of the algorithm and the resultsdemonstrate the potential clinical value of the technique.

  • Abstract
  • Open Access Link
  • Cite
• Journal article
  Vander Poorten E, Tran P, Devreker A, Gruijthuijsen C, Portoles-Diez S, Smoljkic G, Strbac V, Famaey N, Reynaerts D, Vander Sloten J, Tibebu A, Yu B, Rauch C, Bernard F, Kassahun Y, Metzen JH, Giannarou S, Zhao L, Lee S, Yang G, Mazomenos E, Chang P, Stoyanov D, Kvasnytsia M, Van Deun J, Verhoelst E, Sette M, Di Iasio A, Leo G, Hertner F, Scherly D, Chelini L, Häni N, Seatovic D, Rosa B, De Praetere H, Herijgers Pet al., 2016,

  Cognitive Autonomous Catheters Operating in Dynamic Environments

  , Journal of Medical Robotics Research, Vol: 01, ISSN: 2424-905X

  Advances in miniaturized surgical instrumentation are key to less demanding and safer medical interventions. In cardiovascular procedures interventionalists turn towards catheter-based interventions, treating patients considered unfit for more invasive approaches. A positive outcome is not guaranteed. The risk for calcium dislodgement, tissue damage or even vessel rupture cannot be eliminated when instruments are maneuvered through fragile and diseased vessels. This paper reports on the progress made in terms of catheter design, vessel reconstruction, catheter shape modeling, surgical skill analysis, decision making and control. These efforts are geared towards the development of the necessary technology to autonomously steer catheters through the vasculature, a target of the EU-funded project Cognitive AutonomouS CAtheters operating in Dynamic Environments (CASCADE). Whereas autonomous placement of an aortic valve implant forms the ultimate and concrete goal, the technology of individual building blocks to reachsuch ambitious goal is expected to be much sooner impacting and assisting interventionalists in their daily clinical practice.

  • Abstract
  • Open Access Link
  • Cite
• Journal article
  Zhao L, Giannarou S, Lee S, Yang GZet al., 2016,

  SCEM+: real-time robust simultaneous catheter and environment modeling for endovascular navigation

  , IEEE Robotics and Automation Letters, Vol: 1, Pages: 961-968, ISSN: 2377-3766

  Endovascular procedures are characterised by significant challenges mainly due to the complexity in catheter control and navigation. Real-time recovery of the 3-D structure of the vasculature is necessary to visualise the interaction between the catheter and its surrounding environment to facilitate catheter manipulations. State-of-the-art intraoperative vessel reconstruction approaches are increasingly relying on nonionising imaging techniques such as optical coherence tomography (OCT) and intravascular ultrasound (IVUS). To enable accurate recovery of vessel structures and to deal with sensing errors and abrupt catheter motions, this letter presents a robust and real-time vessel reconstruction scheme for endovascular navigation based on IVUS and electromagnetic (EM) tracking. It is formulated as a nonlinear optimisation problem, which considers the uncertainty in both the IVUS contour and the EM pose, as well as vessel morphology provided by preoperative data. Detailed phantom validation is performed and the results demonstrate the potential clinical value of the technique.

  • Abstract
  • Publisher Web Link
  • Open Access Link
  • Cite
• Conference paper
  Zhao L, Giannarou S, Lee S, Merrifield R, Yang GZet al., 2016,

  Intra-operative simultaneous catheter and environment modelling for endovascular navigation based on intravascular ultrasound, electromagnetic tracking and pre-operative data

  , The Hamlyn Symposium on Medical Robotics, Publisher: The Hamlyn Symposium on Medical Robotics, Pages: 76-77
  • Publisher Web Link
  • Open Access Link
  • Cite
• Journal article
  Giannarou S, Ye M, Gras G, Leibrandt K, Marcus HJ, Yang GZet al., 2016,

  Vision-based deformation recovery for intraoperative force estimation of tool–tissue interaction for neurosurgery

  , International Journal of Computer Assisted Radiology and Surgery, Vol: 11, Pages: 929-936, ISSN: 1861-6410

  Purpose: In microsurgery, accurate recovery of the deformation of the surgical environment is important for mitigating the risk of inadvertent tissue damage and avoiding instrument maneuvers that may cause injury. The analysis of intraoperative microscopic data can allow the estimation of tissue deformation and provide to the surgeon useful feedback on the instrument forces exerted on the tissue. In practice, vision-based recovery of tissue deformation during tool–tissue interaction can be challenging due to tissue elasticity and unpredictable motion.Methods: The aim of this work is to propose an approach for deformation recovery based on quasi-dense 3D stereo reconstruction. The proposed framework incorporates a new stereo correspondence method for estimating the underlying 3D structure. Probabilistic tracking and surface mapping are used to estimate 3D point correspondences across time and recover localized tissue deformations in the surgical site.Results: We demonstrate the application of this method to estimating forces exerted on tissue surfaces. A clinically relevant experimental setup was used to validate the proposed framework on phantom data. The quantitative and qualitative performance evaluation results show that the proposed 3D stereo reconstruction and deformation recovery methods achieve submillimeter accuracy. The force–displacement model also provides accurate estimates of the exerted forces.Conclusions: A novel approach for tissue deformation recovery has been proposed based on reliable quasi-dense stereo correspondences. The proposed framework does not rely on additional equipment, allowing seamless integration with the existing surgical workflow. The performance evaluation analysis shows the potential clinical value of the technique.

  • Abstract
  • Publisher Web Link
  • Open Access Link
  • Cite
• Journal article
  Ye M, Giannarou S, Meining A, Yang G-Zet al., 2016,

  Online tracking and retargeting with applications to optical biopsy in gastrointestinal endoscopic examinations

  , Medical Image Analysis, Vol: 30, Pages: 144-157, ISSN: 1361-8415

  With recent advances in biophotonics, techniques such as narrow band imaging, confocal laser endomicroscopy, fluorescence spectroscopy, and optical coherence tomography, can be combined with normal white-light endoscopes to provide in vivo microscopic tissue characterisation, potentially avoiding the need for offline histological analysis. Despite the advantages of these techniques to provide online optical biopsy in situ, it is challenging for gastroenterologists to retarget the optical biopsy sites during endoscopic examinations. This is because optical biopsy does not leave any mark on the tissue. Furthermore, typical endoscopic cameras only have a limited field-of-view and the biopsy sites often enter or exit the camera view as the endoscope moves. In this paper, a framework for online tracking and retargeting is proposed based on the concept of tracking-by-detection. An online detection cascade is proposed where a random binary descriptor using Haar-like features is included as a random forest classifier. For robust retargeting, we have also proposed a RANSAC-based location verification component that incorporates shape context. The proposed detection cascade can be readily integrated with other temporal trackers. Detailed performance evaluation on in vivo gastrointestinal video sequences demonstrates the performance advantage of the proposed method over the current state-of-the-art.

  • Abstract
  • Publisher Web Link
  • Open Access Link
  • Cite
• Journal article
  Kassahun Y, Yu B, Tibebu AT, Stoyanov D, Giannarou S, Metzen JH, Poorten EVet al., 2016,

  Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions (vol 11, pg 553, 2016)

  , International Journal of Computer Assisted Radiology and Surgery, Vol: 11, Pages: 847-847, ISSN: 1861-6410
  • Cite
• Journal article
  Kassahun Y, Yu B, Tibebu AT, Stoyanov D, Giannarou S, Metzen JH, Vander Poorten Eet al., 2015,

  Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions

  , International Journal of Computer Assisted Radiology and Surgery, Vol: 11, Pages: 553-568, ISSN: 1861-6410

  PurposeAdvances in technology and computing play an increasingly important role in the evolution of modern surgical techniques and paradigms. This article reviews the current role of machine learning (ML) techniques in the context of surgery with a focus on surgical robotics (SR). Also, we provide a perspective on the future possibilities for enhancing the effectiveness of procedures by integrating ML in the operating room.MethodsThe review is focused on ML techniques directly applied to surgery, surgical robotics, surgical training and assessment. The widespread use of ML methods in diagnosis and medical image computing is beyond the scope of the review. Searches were performed on PubMed and IEEE Explore using combinations of keywords: ML, surgery, robotics, surgical and medical robotics, skill learning, skill analysis and learning to perceive.ResultsStudies making use of ML methods in the context of surgery are increasingly being reported. In particular, there is an increasing interest in using ML for developing tools to understand and model surgical skill and competence or to extract surgical workflow. Many researchers begin to integrate this understanding into the control of recent surgical robots and devices.ConclusionML is an expanding field. It is popular as it allows efficient processing of vast amounts of data for interpreting and real-time decision making. Already widely used in imaging and diagnosis, it is believed that ML will also play an important role in surgery and interventional treatments. In particular, ML could become a game changer into the conception of cognitive surgical robots. Such robots endowed with cognitive skills would assist the surgical team also on a cognitive level, such as possibly lowering the mental load of the team. For example, ML could help extracting surgical skill, learned through demonstration by human experts, and could transfer this to robotic skills. Such intelligent surgical assistance would significantly surpass the st

  • Abstract
  • Cite
• Journal article
  Keir GJ, Nair A, Giannarou S, Yang G-Z, Oldershaw P, Wort SJ, MacDonald P, Hansell DM, Wells AUet al., 2015,

  Pulmonary vasospasm in systemic sclerosis: noninvasive techniques for detection

  , Pulmonary Circulation, Vol: 5, Pages: 498-505, ISSN: 2045-8940

  In a subgroup of patients with systemic sclerosis (SSc), vasospasm affecting the pulmonary circulation may contribute to worsening respiratory symptoms, including dyspnea. Noninvasive assessment of pulmonary blood flow (PBF), utilizing inert-gas rebreathing (IGR) and dual-energy computed-tomography pulmonary angiography (DE-CTPA), may be useful for identifying pulmonary vasospasm. Thirty-one participants (22 SSc patients and 9 healthy volunteers) underwent PBF assessment with IGR and DE-CTPA at baseline and after provocation with a cold-air inhalation challenge (CACh). Before the study investigations, participants were assigned to subgroups: group A included SSc patients who reported increased breathlessness after exposure to cold air (n = 11), group B included SSc patients without cold-air sensitivity (n = 11), and group C patients included the healthy volunteers. Median change in PBF from baseline was compared between groups A, B, and C after CACh. Compared with groups B and C, in group A there was a significant decline in median PBF from baseline at 10 minutes (−10%; range: −52.2% to 4.0%; P < 0.01), 20 minutes (−17.4%; −27.9% to 0.0%; P < 0.01), and 30 minutes (−8.5%; −34.4% to 2.0%; P < 0.01) after CACh. There was no significant difference in median PBF change between groups B or C at any time point and no change in pulmonary perfusion on DE-CTPA. Reduction in pulmonary blood flow following CACh suggests that pulmonary vasospasm may be present in a subgroup of patients with SSc and may contribute to worsening dyspnea on exposure to cold.

  • Abstract
  • Open Access Link
  • Cite
• Journal article
  Shen M, Giannarou S, Yang G-Z, 2015,

  Robust camera localisation with depth reconstruction for bronchoscopic navigation

  , International Journal of Computer Assisted Radiology and Surgery, Vol: 10, Pages: 801-813, ISSN: 1861-6410

  PurposeBronchoscopy is a standard technique for airway examination, providing a minimally invasive approach for both diagnosis and treatment of pulmonary diseases. To target lesions identified pre-operatively, it is necessary to register the location of the bronchoscope to the CT bronchial model during the examination. Existing vision-based techniques rely on the registration between virtually rendered endobronchial images and videos based on image intensity or surface geometry. However, intensity-based approaches are sensitive to illumination artefacts, while gradient-based approaches are vulnerable to surface texture.MethodsIn this paper, depth information is employed in a novel way to achieve continuous and robust camera localisation. Surface shading has been used to recover depth from endobronchial images. The pose of the bronchoscopic camera is estimated by maximising the similarity between the depth recovered from a video image and that captured from a virtual camera projection of the CT model. The normalised cross-correlation and mutual information have both been used and compared for the similarity measure.ResultsThe proposed depth-based tracking approach has been validated on both phantom and in vivo data. It outperforms the existing vision-based registration methods resulting in smaller pose estimation error of the bronchoscopic camera. It is shown that the proposed approach is more robust to illumination artefacts and surface texture and less sensitive to camera pose initialisation.ConclusionsA reliable camera localisation technique has been proposed based on depth information for bronchoscopic navigation. Qualitative and quantitative performance evaluations show the clinical value of the proposed framework.

  • Abstract
  • Author Web Link
  • Open Access Link
  • Cite

This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.