Pathological Site Retargeting under Tissue Deformation using Geometrical Association and Tracking

Size: px

Start display at page:

Download "Pathological Site Retargeting under Tissue Deformation using Geometrical Association and Tracking"

Hugo Bryan
8 years ago
Views:

1 Pathological Site Retargeting under Tissue Deformation using Geometrical Association and Tracking Menglong Ye 1, Stamatia Giannarou 1, Nisha Patel 1,2, Julian Teare 1, and Guang-Zhong Yang Department of Surgery and Cancer Imperial College London

2 Gastrointestinal (GI) Endomicroscopy Advantages of pcle In vivo, in situ, real-time tissue characterisation and assessment Non-invasive tissue biopsies Early diagnosis of diseases such as cancer Confocal mini-probes (courtesy of Mauna Kea Technologies) Limitations of pcle Difficult to systematically scan a large organ surface No landmarks left on the tissue surface Difficult to retarget optical biopsy sites (pathological sites) Probe-based Gastrointestinal Confocal (GI) Laser endoscopy Endomicroscopy (courtesy of Prof. Meining) (pcle) procedure (courtesy of Mauna Kea Technologies)

3 Previous Work on Retargeting Requirements of pathological site retargeting: Robustness to tissue deformation The ability of online retargeting during the same procedure (Mountney et al., 2009) Robustness to partial occlusion during pcle procedures (Allain et al., 2012) Mountney et al. Optical Biopsy Mapping for Minimally Invasive Cancer Screening. MICCAI, 2009, 5761, Allain et al. Re-localisation of a biopsy site in endoscopic images and characterisation of its uncertainty. Med. Image Anal., 2012, 16,

4 Overview of the Approach Local Affine Tissue Deformation Modelling Tracking with Cascaded Online Learning Geometrical Association for Retargeting Flow chart of the proposed approach for pathological site retargeting

5 Local Affine Tissue Deformation Modelling The We estimate application the of transformation computer vision of the techniques target site in minimally the initial invasive view procedures and current is challenging view, and this due can to be tissue achieved deformation by tracking its surrounding regions (these regions are inside the local affine patch) A deforming surface can be modelled using local affine patches The relation of the positions of an affine patch in two different views can be defined as a homography H As small image patches are considered, H can be estimated using an affine transformation

6 Tracking with Cascaded Online Learning To track surrounding regions, tracking-learning-detection (TLD) is adopted It is a tracking method that combines cascaded online learning and multi-scale detection It provides robust tracking and online re-identification of a target Multiple surrounding regions are independently tracked Flow diagram of TLD (Kalal et al., 2012) Kalal et al. Tracking-learning-detection IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34,

7 Geometrical Association for Retargeting The target site positions in the current image and initial image are related using a local affine transformation (robust to deformation) Affine transformations are estimated using Least-Squares, which requires corresponding point pairs These corresponding points are located at the centres of the surrounding tracked regions Pentagon structures (rotated through different angles) are used to organise the surrounding regions for tracking Ensure enough point pairs for Least-Squares Non-collinearity for affine transformation estimation r1 θ 1 st Level The contact point of the probe and tissue surface

8 Geometrical Association for Retargeting For the current image, one pentagon gives a candidate target site position, when k (k 4) regions are identified by TLD: r1 θ A collection of candidate positions can be derived: 1 st Level The final target site position in the current image is chosen as the median values of all the candidates Umeyama, S. Least-squares estimation of transformation parameters between two point patterns IEEE Trans. Pattern Anal. Mach. Intell., 1991, 13,

9 Geometrical Association for Retargeting When the probe is detected to have moved away from the target site, we re-initialise three levels of pentagons with different radii Increase the number of candidate positions Allow larger extents of zoom-out and zoom-in As previously, the final position in the current image is chosen as the median values of all the candidates 3 rd level 1 st level 2 nd level r1 θ 1 st Level

10 Olympus NBI and Pentax i-scan Endoscopy Olympus NBI Pentax i-scan

11 Experiment Results Phantom data Video 1: Deforming phantom Video 3: simulated pcle procedures Video 2: Camera rotation

12 Experiment Results In vivo data Video 4: in vivo data (Olympus NBI) Video 5: in vivo data (Pentax i-scan)

13 Experiment Results Validation Phantom data. Precision values: Sensitivity(recall) values: Specificity values: In vivo data. Precision values: Sensitivity(recall) values: Specificity values:

14 Limitations and Recent Extension Current method relies on tissue tracking, so it would fail when the tissue surface is completely uniform. It allows a certain degree of out-of-plane rotation. Presented method Enhanced method using geometrical boosting

15 Time Complexity and Run-time Analysis Time complexity analysis (n is the number of the surrounding regions for tracking): Tracking: O(n) Detection: O(1) Learning: O(n) 500 video frames of size 720x576 were processed Run-time analysis Intel i5 dual-core 2.53GHz Intel Xeon quad-core 2.66Ghz Intel Xeon Phi coprocessor (64 cores) will be used for real-time performance

16 Conclusions and Future work An online pathological site retargeting approach has been proposed based on a learning-based tracker A local geometrical association scheme has been introduced to provide robustness to partial occlusion The method provides reliable retargeting of pathological sites in the presence of tissue deformation where existing methods fail Experimental results have demonstrated the accuracy of the method and its potential clinical value in endoscopy Future work will focus on achieving real-time performance, testing on interprocedures, and addressing Barrett s oesophagus

17 Thank you! Poster: O4-03 Acknowledgements: Prof. Alexander Meining, Technische Universität München Mr. Tou-Pin Chang,, Imperial College London

Hybrid Retargeting for High-Speed Targeted Optical Biopsies

Hybrid Retargeting for High-Speed Targeted Optical Biopsies André Mouton 1, Menglong Ye 1, François Lacombe 2, and Guang-Zhong Yang 1 1 The Hamlyn Centre for Robotic Surgery, Imperial College London, UK