CCD cameras and digital Imaging Pascal Chartrand Modern microscopy depends on digital image capture, so we start there There are Various Types of Electronic Cameras: TV cameras of many kinds and cameras that use Charged Coupled Devices (CCDs). Thus, we will discuss: CCD Camera Architecture CCD Camera Performance Acquiring a Digital Image 1
Electronic Detection of Light depends on the photo-electric effect When a photon hits a metal or a doped surface, an electron is energized and therefore destabilized Its presence can be recorded by applying a voltage to pull the energized e - off as a current Detectors with no spatial discrimination Photomultiplier tubes (LSCMs) Photodiodes Detectors with spatial discrimination Tube cameras - Vidicon, Newvicon, SIT, ISIT Solid-state detectors CCDs The sensitivities of various electronic cameras: Video - CCD 2
CCD cameras Chip camera Photodiodes A light-sensitive semi-conductor set up so incident light can knock electrons loose A loose electron leaves behind a zone of positive charge or a hole Electrons and holes can move in response to an electric field Current is then proportional to the number of loose electrons, i.e., to number of incident photons 3
For many LM applications, the best imaging device is a Charge-Coupled Device (CCD) A dense matrix of photodiodes with charge storage regions called wells or pixels. Stored charge can be transferred to a serial register where it is held until an amplifier reads out the quantity of charge accumulated in each pixel. Image from Fundamentals of Light Microscopy and Electronic Imaging, Doug Murphy Diagram of an individual CCD Well or Pixel Hazelwood et al., in Shorte and Frischknecht 2007 full well capacity on the order of ~1000 electrons/µm 2 4
From Video Microscopy, 2 nd Ed., Inoué and Spring A metaphore for CCD camera readout 5
Three Primary CCD Chip Designs From Video Microscopy, 2 nd Ed., Inoué and Spring Parameters of Camera Performance Quantum efficiency Spectral sensitivity Spatial resolution Dynamic range Signal-to-noise ratio Temporal resolution (speed) Linearity What is most important for your experiments? 6
Spectral Sensitivity & Quantum Efficiency Quantum efficiency: percentage of photons hitting the photoreactive surface that will produce an electron hole pair - It is an accurate measurement of the device's electrical sensitivity to light Graph from microscopyu.com Improvements in Interline CCDs Single microlens added Input light Microlens No microlens Open window Image from Butch Moomaw, Hamamatsu 7
Backed thinned CCDs have higher QE From Video Microscopy, 2 nd Ed., Inoué and Spring The idea of digital image sampling Hazelwood et al., in Shorte and Frischknecht 2007 8
Accuracy of digital sampling depends in part on spatial frequency of sampling Nyquist-Shannon criterion and image sampling How many CCD pixels are needed to accurately reproduce the smallest object that can be resolved by the scope? When sampling a signal (e.g., converting from an analog signal to digital) the sampling frequency must be at least twice the highest frequency present in the input signal if you want to reconstruct the original perfectly from the sampled version. 9
Nyquist-Shannon Sampling Optimal Pixel Size for Preserving Resolution - For a CCD camera, photons emitted by a single point source should be captured by at least 3 pixels Objective NA Airy disk radius (µm) Projected size on CCD (µm) Optimal pixel size (µm) 40x 1.3 0.26 10.4 3.5 60x 1.4 0.24 14.4 4.8 100x 1.4 0.24 24.0 8 Pixels range from 4 25µm. Around 7µm is now most common. Additional magnification can be used to optimize camera resolution for a given objective lens. 10
Aliasing Nyquist sampled Undersampled Dynamic Range Dynamic Range is defined as the full well capacity divided by the camera noise Large dynamic range allows discrimination between bright and dim parts of the specimen. Smaller pixels generally have smaller full well capacity. 11
Dynamic Range, bits and gray levels Bit = 2 n gray levels. A 10 bit camera has 2 10 or 1024 gray levels All camera have one goal: Separating SIGNAL from NOISE Signal is defined as the change in the state of a detector produced by photons from the object of interest. Noise is defined as meaningless fluctuations in the signal. It is of three kinds: Optical Noise is defined as any detected photon produced by stray light from the microscope or scattered from the object of interest Camera Noise is defined as any change in the detector output not produced by photons from the object of interest. Statistical Noise is defined as fluctuations in signal that arise from the random changes that result from inadequate sampling. 12
Noise 1. Statistical Noise: Random fluctuations in signal 2. Dark (Thermal) Noise Electrons pop out of the chip as it heats up They build up with long exposure time Can be reduced by cooling chip 3. Read Out Noise Errors as chip is read Constant, regardless of exposure time Can be reduced by reading the chip more slowly 4. Camera Noise (Dark + Readout) 5. Total Noise (Signal noise + Camera noise) 1.25 MHz/pixel 10 MHz/pixel Images collected by JWS in the Nikon Imaging Center at Harvard Medical School Signal to Noise Ratio SNR = S N SNR: Signal to noise ratio S: Signal detected by the camera N: Total noise Improve the Signal by Collecting more photons from your specimen Using a more sensitive camera Amplyfing the signal, so long as it increases the difference between signal and noise Reducing the Noise Reduce camera noise Use frame averaging 13
S/N camera = full well capacity / camera noise S/N image = signal / (signal noise + camera noise) Signal to Noise begins with the Signal Signal = I x QE x T S (electrons captured in pixels) = Signal I (photons/sec) = Input light level QE (electrons/photon) = Quantum efficiency T (sec) = Integration time All signals have statistical noise associated with them Signal SNR = Think of it as counting Signal radioactivity 14
As the signal level increases, the S/N image gets better since the square root becomes a smaller fraction of the signal. This affects the precision of the data you can record. 5 / 5 = 2,24 10,000 / 10,000 = 100 100,000 / 100,000 = 316 Pictorial display of signal-to-noise ratio and its effect on image quality 15
What can YOU do to increase the signal that the camera sees? Brighter fluorophores Align illumination optics (Koehler illumination) & bulb Brighter objective lens (B = NA 4 / M 2 ) Higher transmission objective lens (less aberration correction) Minimize spherical aberration Decrease specimen noise (mounting medium, BG fluorescence) Decrease other optical noise (minimize reflective surfaces, use field diaphragm, work in dark, no dirt) Low Light Level Cameras At low light levels, the signal approaches the noise level. Cameras employ different methods to improve the S/N camera. Low light cameras increase the signal, reduce the noise, or both. Decrease Noise: Cooling, Slower Readout, Frame Averaging Increase Signal: Higher QE, Bigger Pixels, Longer Exposure, Amplification 16
Vacuum Chamber CCD Heat Sink Window Thermo Electric Cooler Cooled CCD Images from Butch Moomaw, Hamamatsu - Cooling the CCD camera reduces the dark current -Use thermo-electric cooling (Peltier effect) to reach -30 C to -40 C) Graph from Video Microscopy, 2 nd Ed., Inoué and Spring Electron Multiplication (On-Chip Multiplication) - Signal amplification occurs in the gain register, before readout - Electrons are accelerated from pixel to pixel, generating secondary eletrons by impact ionization (A) to (B) - Camera cooled to -80 C (decrease dark noise) - However, signal amplification also increases dark noise Image from www.andor-tech.com 17
EM-CCD Performance ORCA- ER 500ms, High Gain EM-CCD 200ms, Low Multiplication EM-CCD 200ms, Med Multiplication EM-CCD 50ms, High Multiplication Images collected by JWS in the Nikon Imaging Center at Harvard Medical School Tips on acquiring a digital image Determine optimal exposure time by looking at the gray scale values!! For fixed specimens, use full dynamic range. For live cells, collect as many photons as you can, considering temporal resolution, spatial resolution, bleaching, toxicity, etc. CCDs and PMTs are linear until saturated. 18
Do not judge image quality based on how the image looks on the monitor 256 Gray levels 64 Gray levels 16 Gray levels Images from Butch Moomaw, Hamamatsu Image Scaling The monitor can only display 8 bits. Your image is probably 12 or 14 bits. Scaling determines how the data are displayed on the monitor. Actual gray values 198-1265 Displaying 0-4095 Actual gray values 198-1265 Displaying 198-1265 Auto-scaled Actual gray values 198-1265 Displaying 236-546 Images collected by JWS in the Nikon Imaging Center at Harvard Medical School 19
Exposure Time 50ms 100ms 200ms 400ms 500ms 687 1051 1858 3260 3888 Image maximum grayscale value 12-bit camera maximum = 4095 Increasing exposure time increases signal while decreasing temporal resolution Images collected by JWS in the Nikon Imaging Center at Harvard Medical School Gain Gain: the number of accumulated photoelectrons that determine each gray level step distinguished by the readout electronics Increasing gain reduces the number of photons /gray scale value Example: if your gain is 8 ē per gray value, a pixel signal of 8000 ē correspond to 1000 gray levels. Increasing the gain by 4 gives you 2 ē per gray value, and the same pixel signal results in 4000 gray levels Increasing gain, Same exposure time Images collected by JWS in the Nikon Imaging Center at Harvard Medical School 20
Binning (2x2) Read out 4 pixels as one Increases SNR by 2x Decreases read time by 2 or 4x Decreases resolution by 2x Binning 50ms, 2x2 bin Max 2297 25 ms, 4x4 bin Max 3493 50ms, no binning Max 746 Images collected by JWS in the Nikon Imaging Center at Harvard Medical School 21
Reading the manufacturer specifications Important informations are provided by the manufacturer! Photometrics CoolSnap fx Reading the manufacturer specifications Camera noise (read noise and dark current) is given in electrons. Gray scale values are converted to electrons by multiplying by an electron conversion factor. The electron conversion factor = full well capacity / max gray scale value. Photometrics CoolSnap fx 22
23