Medical Imaging, Acquisition and Processing

Transcription

1 Medical Imaging, Acquisition and Processing J. Kybic Center for Machine Perception, CTU Prague Medical Imaging, Acquisition and Processing p.1/59

2 Overview Introduction and motivation Modalities (acquisition devices) Microscopy, X-rays, CT, Ultrasound, PET/SPECT, EEG/MEG,... Data processing Preprocessing, Reconstruction, Segmentation, Registration, Classification,... Conclusions Medical Imaging, Acquisition and Processing p.2/59

3 Medical imaging pipeline Medical Imaging, Acquisition and Processing p.3/59

4 Medical imaging pipeline example patient/subject Physical property Raw data Improved data Images/volumes Quantitative/qualitative inf. Imaging device Preprocessing Reconstruction Interpretation physician Decision Medical Imaging, Acquisition and Processing p.4/59

5 Medical imaging pipeline example patient/subject Physical property Raw data Improved data Images/volumes Quantitative/qualitative inf. Imaging device Preprocessing Reconstruction Interpretation physician Decision Medical Imaging, Acquisition and Processing p.4/59

6 Medical imaging pipeline example patient/subject Physical property Raw data Improved data Images/volumes Quantitative/qualitative inf. Statistical processing Imaging device Preprocessing Reconstruction Interpretation physician Decision Medical Imaging, Acquisition and Processing p.4/59

7 Imaging device Physical property: density, absorbance, conductivity,... Imaging device Raw data Examples: CT, X-rays, MRI, ultrasound,... Medical Imaging, Acquisition and Processing p.5/59

8 Preprocessing A priori knowledge Raw data Preprocessed, cleaned, enhanced data Preprocessing Examples: Noise suppression, contract enhancement, intensity equalization, outlier elimination, bias compensation, time/space filtering,... Medical Imaging, Acquisition and Processing p.6/59

9 Reconstruction Physics of the acquisition device Preprocessed raw data Reconstruction Physically meaningfu images, volumes, sequences Inverse problem Examples: Tomographic reconstruction MRI, ultrasound... Medical Imaging, Acquisition and Processing p.7/59

10 Interpretation A priori knowledge Images, volumes, sequences Examples: Interpretation High-level processing Quantitative & qualitative inf. size, changes, metabolic inf. Registration, segmentation, classification, tracking,... Medical Imaging, Acquisition and Processing p.8/59

11 Modalities Medical Imaging, Acquisition and Processing p.9/59

12 Microscopy Optical microscopy since 17th century; Jensen, van Leeuwenhoek, Galilei,... Medical Imaging, Acquisition and Processing p.10/59

13 Microscopy examples placenta cross-section Medical Imaging, Acquisition and Processing p.11/59

14 Microscopy examples muscle capillaries Medical Imaging, Acquisition and Processing p.11/59

15 Microscopy examples crocodile ear slice Medical Imaging, Acquisition and Processing p.11/59

16 Microscopy examples retina Medical Imaging, Acquisition and Processing p.11/59

17 Microscopy types & trends Electron microscopy Electron transmission microscopy Confocal microscopy reject out-of-focus light, scanning Phase-contrast microscopy Fluorescence microscopy fluorescent dyes multiple sensing channels/filters multiple light sources visible, UV CCD cameras supercooled superresolution Moveable specimen tray Auto-focussing Automated acquisition, mosaicking Medical Imaging, Acquisition and Processing p.12/59

18 Microscopy Advantages High-spatial resolution Colour and texture information Affordable (optical microscopy) Proven technique large body of experts available Disadvantages Difficulties of in-vivo observations Inherently 2D Missing large-scale perspective Medical Imaging, Acquisition and Processing p.13/59

19 X-rays / Radiography 1895, W. Röntgen B. Röntgen hand modern hand Medical Imaging, Acquisition and Processing p.14/59

20 X-rays / Radiography (2) Modern chest radiography machine Medical Imaging, Acquisition and Processing p.15/59

21 X-rays / Radiography (3) Chest X-ray: Medical Imaging, Acquisition and Processing p.16/59

22 X-rays / Radiography (4) Advantages Widely used and available Experts available High-spatial resolution Excelent imaging of hard tissues (bones) Disadvantages Radiation exposure Difficulty in imaging soft-tissues 2D projection, hidden parts Medical Imaging, Acquisition and Processing p.17/59

23 X-rays / Radiography (5) New trends CCD sensors replace film higher sensitivity, faster exposure, lower dose dynamic imaging Medical Imaging, Acquisition and Processing p.18/59

24 Computed tomography The machine: Medical Imaging, Acquisition and Processing p.19/59

25 Computed tomography (2) Principle:? Medical Imaging, Acquisition and Processing p.20/59

26 Computed tomography (3) Example, lungs: Medical Imaging, Acquisition and Processing p.21/59

27 Computed tomography (3) Example, head: Medical Imaging, Acquisition and Processing p.21/59

28 Computed tomography (4) Advantages 3D volume High-spatial resolution Disadvantages Increased radiation dose Longer acquisition times Shadows, difficulties imaging soft-tissues Costly equipment Medical Imaging, Acquisition and Processing p.22/59

29 Ultrasound Equipment Medical Imaging, Acquisition and Processing p.23/59

30 Ultrasound Equipment Medical Imaging, Acquisition and Processing p.23/59

31 Ultrasound (2) Examples Fetus Medical Imaging, Acquisition and Processing p.24/59

32 Ultrasound (2) Examples Heart Medical Imaging, Acquisition and Processing p.24/59

33 Ultrasound (2) Examples Heart, Doppler Medical Imaging, Acquisition and Processing p.24/59

34 Ultrasound (3) Advantages Low-cost Proven technique Experts available Not invasive, no harmful effects Good imaging of soft tissues Easy dynamic acquisition Disadvantages Low-signal quality, speckle noise Low penetration depth or low spatial resolution Shadows from bones and other thick tissues Mostly 2D Medical Imaging, Acquisition and Processing p.25/59

35 Ultrasound (4) New trends 3D ultrasound High-frequency ultrasound Doppler ultrasound Medical Imaging, Acquisition and Processing p.26/59

36 MEG and EEG Magnetoencephalography Electroencephalography Medical Imaging, Acquisition and Processing p.27/59

37 MEG sensors SQUID array Gradiometers Medical Imaging, Acquisition and Processing p.28/59

38 Currents in the brain Quadripolar structure of action potential Action Potential 100 mv Propagation -80 mv K+ 1 ms + 20 mv Na+ -80mV Axon Membrane K+ Ion-pump Repolarization Na+ Ion-pump Depolarization Primary currents J p [A/m 2 ] Medical Imaging, Acquisition and Processing p.29/59

39 Magnetic measurements Medical Imaging, Acquisition and Processing p.30/59

40 Source localization Surface field: Brain field: Medical Imaging, Acquisition and Processing p.31/59

41 MEG and EEG Advantages EEG widely available Excellent temporal resolution, can observe brain activities at ms resolution Disadvantages MEG extremely expensive Poor spatial sampling / resolution Good reconstruction algorithms not yet available Necessity of a good head model Medical Imaging, Acquisition and Processing p.32/59

42 Magnetic resonance imaging (MRI) Larmor, Bloch, Purcell, Rabi, Lauterbur,... around Medical Imaging, Acquisition and Processing p.33/59

43 MRI Principles ω = γb Medical Imaging, Acquisition and Processing p.34/59

44 MRI Machine from inside Medical Imaging, Acquisition and Processing p.35/59

45 MRI Example Brain slice: Medical Imaging, Acquisition and Processing p.36/59

46 MRI Advantages Good spatial resolution True 3D acquisition Excellent contrast for all tissue types In vivo, non-invasive, no radiation Disadvantages Very expensive Acquisition somewhat long (3D) Poor temporal resolution (3D) Patient discomfort noise, claustrophoby Strong magnetic field Medical Imaging, Acquisition and Processing p.37/59

47 MRI Types and trends Sequences: gradient-echo, spin-echo,... Weighting: T1, T2, PD, BOLD,... Acceleration: EPI, FLASH, parallel,... Modalities: fmri, perfusion, flow,... Medical Imaging, Acquisition and Processing p.38/59

48 Other imaging methods Positron emission tomography (PET) Single positron emission tomography (SPECT) Gamma camera Termography... Medical Imaging, Acquisition and Processing p.39/59

49 Medical data processing Medical Imaging, Acquisition and Processing p.40/59

50 Medical imaging specificities (wrt. Computer vision) We want the physical reality, not to emulate human perception. We deal with many modalities, optical intuition often does not apply. Data are rare and expensive maximum information must be extracted. Approximations to be avoided if possible. Data are continuous, result is continuous we do not decide, we help to decide Algorithms must be robust and safe give no answer or a conservative answer, rather than a wrong answer. Physician is the ultimate judge computer is a tool. Accountability and verifyiability help credibility. Medical comunity wants clinical trials. Medical Imaging, Acquisition and Processing p.41/59

51 Basic data processing tasks Low level operations Segmentation Registration Classification Reconstruction / Inverse problems... Medical Imaging, Acquisition and Processing p.42/59

52 Contract enhancement original enhanced Medical Imaging, Acquisition and Processing p.43/59

53 Edge detection retinal cells Medical Imaging, Acquisition and Processing p.44/59

54 Segmentation Divide image into classes. Medical Imaging, Acquisition and Processing p.45/59

55 Segmentation (2) Easy for humans, difficult for computers. We do not know how brain perform segmentation. Algorithm is image and task dependent, needs a priori knowledge. Useful for: Separating objects, masking background Image simplification Shape, size and other statistics Medical Imaging, Acquisition and Processing p.46/59

56 How do we segment? Cells are black. Medical Imaging, Acquisition and Processing p.47/59

57 How do we segment? Cells are homogeneous connected components. Medical Imaging, Acquisition and Processing p.47/59

58 How do we segment? Arteries are bright and elongated. Medical Imaging, Acquisition and Processing p.47/59

59 Segmentation BRAIN MRI, SEGMENTATION fromresult MRI (T1) 10/25 original, gray and white brain matter, CSF Medical Imaging, Acquisition and Processing p.48/59

60 Image registration Find corresponding points Manual, automatic, semi-automatic Useful for: Comparing images from different times Comparing images from different methods Comparing images from different subjects Analyzing movement Segmentation Provides qualitative and quantitative information. Medical Imaging, Acquisition and Processing p.49/59

61 Image registration (2) reference image registration test image deformation warping warped image Medical Imaging, Acquisition and Processing p.50/59

62 Image alignment reference test before Medical Imaging, Acquisition and Processing p.51/59

63 Image alignment reference test before warped Medical Imaging, Acquisition and Processing p.51/59

64 Manual registration Landmark identification Landmark interpolation Medical Imaging, Acquisition and Processing p.52/59

65 Registration as minimization difference reference image criterion E optimization deformation function g(x) deformation test image deformed image Medical Imaging, Acquisition and Processing p.53/59

66 Registration examples EPI distortion After Medical Imaging, Acquisition and Processing p.54/59

67 Registration examples EPI distortion MRI atlas Atlas Medical Imaging, Acquisition and Processing p.54/59

68 Registration examples EPI distortion MRI atlas CT alignment After Medical Imaging, Acquisition and Processing p.54/59

69 Registration examples EPI distortion MRI atlas CT alignment SPECT atlas (with University Hospital in Geneva) Medical Imaging, Acquisition and Processing p.54/59

70 Registration examples EPI distortion MRI atlas CT alignment SPECT atlas Ultrasound velocity (with María J. Ledesma-Carbayo) Medical Imaging, Acquisition and Processing p.54/59

71 Registration examples EPI distortion MRI atlas CT alignment SPECT atlas Ultrasound MRI heart sequence Medical Imaging, Acquisition and Processing p.54/59

72 Registration examples EPI distortion MRI atlas CT alignment SPECT atlas Ultrasound MRI heart sequence MRI perfusion original uncorrected corrected Medical Imaging, Acquisition and Processing p.54/59

73 Tomographic reconstruction? The problem Medical Imaging, Acquisition and Processing p.55/59

74 Tomographic reconstruction Solution Medical Imaging, Acquisition and Processing p.55/59

75 Tomographic reconstruction Another solution Medical Imaging, Acquisition and Processing p.55/59

76 Radon transform s i,1 s i,2 u s i,j t angle θ i (Rf)(θ i, u j ) = f ( t cos θ i u j sin θ i, t sin θ i + u j cos θ i ) dt Medical Imaging, Acquisition and Processing p.56/59

77 Radon transform s i,1 s i,2 u s i,j t angle θ i (Rf)(θ i, u j ) = f ( ) t cos θ i u j sin θ i, t sin θ i + u j cos θ i dt ( ) I = I 0 exp ρ(r)ds(r) r S Medical Imaging, Acquisition and Processing p.56/59

78 Reconstruction algorithms Phantom Phantom Medical Imaging, Acquisition and Processing p.57/59

79 Reconstruction algorithms Backprojection Backprojection nt=8 nu=32 50 Variational reconstruction Variational reconstruction nu=32 nt=8 gamma= angles (each 22 ), 32 samples per angle Medical Imaging, Acquisition and Processing p.57/59

80 Other algorithms and topics Inverse problems MEG, optical tomography,... Classification is the patient/sample healthy? Optical, shape reconstruction provide 3D shape Texture analysis Machine learning, uncertainty Related subjects: Databases and information infrastructure PACS (Picture Archiving and Communication Systems) Expert systems, knowledge engineering, data mining... Medical Imaging, Acquisition and Processing p.58/59

81 Medical imaging Conclusions Medical imaging experiences a recent boom: Many modalities Instruments are digitalized and more widespread More powerful and affordable computers Large quantity of data is produced. Computers can help to: make data meaningful extract previously unavailable information visualize, compare and analyse scan for abnormalities provide the right information for physicians to make the decision Medical Imaging, Acquisition and Processing p.59/59

82 Medical imaging Conclusions Medical imaging experiences a recent boom: Many modalities Instruments are digitalized and more widespread More powerful and affordable computers Large quantity of data is produced. Computers can help to: make data meaningful extract previously unavailable information visualize, compare and analyse scan for abnormalities provide the right information for physicians to make the decision Medical Imaging, Acquisition and Processing p.59/59