Cameras for Optical Microscopy

Transcription

1 Cameras for Optical Microscopy

2 Charge Coupled Device (CCD) Cameras The fundamental processes involved in creating an image with a CCD camera include: exposure of the photodiode array elements to incident light conversion of accumulated photons to electrons organization of the resulting electronic charge in potential wells transfer of charge packets through the shift registers to the output amplifier Charge output from the shift registers is converted to voltage and amplified prior to digitization in the A/D converter.

3 Light-sensing unit of the CCD is a metal oxide semiconductor (MOS) capacitor operated as a photodiode and storage device. The substrate is a p/n type silicon wafer insulated with a thin layer of silicon dioxide (approximately 100 nanometers). Pixels, are defined in the silicon matrix by an orthogonal grid of narrow transparent currentcarrying electrode strips, or gates that are used to control the collection and transfer of photoelectrons. Electrons are liberated by photon interaction on a thin transparent silicon layer. With reverse bias operation, negatively charged electrons migrate to an area underneath the positively charged gate electrode (potential well). Individual pixels are isolated from their neighbors by insulating barriers, or channel stops. Charges are then transferred to a neighboring pixel by controlling the gate voltage.

4 At the end of the integration period, accumulated charge in pixels is shifted row by row across the parallel register which is then transferred into the serial shift register. Charge contents of pixels are transferred into an output node to be read by an onchip amplifier, which boosts the electron signal and converts it into an analog voltage output. An ADC assigns a digital value for each pixel according to its voltage amplitude. Each pixel value is stored in computer memory and the complete image file is displayed for visual evaluation. The CCD is cleared of residual charge prior to the next exposure by executing the full readout cycle except for the digitization step.

5

6 CCD Architecture Full Frame: Nearly 100% photosensitive, no dead space between pixels Time resolution is limited by readout speed (dead time) Frame Transfer: One half of the chip is masked and used for storage. Photon accumulation and readout can be done simultaneously. Interline Transfer: Charges can be transferred to the adjacent pixel. Faster shift. Lower spatial resolution and lost signal. Adherent microlenses can be used to increase photosensitive area by 75%.

7 Quantum Efficiency of CCDs The losses due to gate channel structures are completely eliminated in the back-illuminated CCD. In this design, the back of the CCD has been thinned (10-15 microns) by etching until it is transparent nanometer range have a relatively high absorption coefficient in silicon. Front illuminated CCDs through the gate electrodes and oxide coatings, are more sensitive between 550 and 900 nanometers.

8 Signal (S) is determined as a product of input light level (I), quantum efficiency (QE) and the integration time (t) measured in seconds. S = I QE T The primary sources of noise considered in determining the ratio are statistical (shot noise), thermal noise (dark current) and preamplification or readout noise, SNR = IQ e t / [ IQ e t + Dt + N r 2 ] 1/2 I the incident photon flux (photons/pixel/second), D the dark current (electrons/pixel/second), and N(r) represents read noise (electrons rms/pixel/image). At a low light regime, signal must be multiplied to improve SNR!

9 Intensified CCDs Incoming photons are converted to electrons in the photocathode. Electron output is amplified in the microchannel plate. Amplified electrons are accelerated by a high potential difference onto a phosphorescent screen that converts the electrons to photons. Fluorescent signal is projected onto individual pixels on a CCD array by a fiber optic tapered bundle. The resolution is ultimately limited by the photocathode, the micro-channel plate, and the output phosphor. 50% of fluorescence signal spreads over to neighboring pixels

10 Electron Multiplied CCDs EMCCD is capable of detecting single photon events whilst maintaining high Quantum Efficiency, achievable by way of a unique electron multiplying structure built into the sensor. Unlike a conventional CCD, an EMCCD is not limited by the readout noise of the output amplifier The EM register has several hundred stages that use higher than normal clock voltages. As charge is transferred through each stage the phenomenon of Impact Ionization is utilized to produce secondary electrons, and hence EM gain.

11 R1 and R3) are clocked with drive pulses of normal potential, which is typically on the order of 5 to 15 volts (the R3 gates have zero potential for the clocking phase. R2 is clocked at higher voltage (35-50 volts) preceded by a gate held at a low DC level The potential difference sustains the impact ionization process as electrons are transferred from phase 1 to phase 2 in the normal clocking sequence.

12 The multiplication gain is exponentially proportional to the applied high phase-2 voltage, and can be increased or decreased by varying the clock voltages. M = (1 + g) N where g (0.013) is the probability of generating a secondary electron and N (512) is the number of pixels in the multiplication register. Total charge multiplication gain is over 744.

13 Parameters in Digital Imaging Quantum Efficiency: Probability of generating photoelectron out of incoming photon. Dark Current: Spontaneous generation of electron due to thermal noise. Spatial Resolution: Determines the ability to capture fine specimen details without pixels being visible in the image. Effective Pixel Size: Actual camera pixel size divided by magnification. Nyquist Criterion, 100 nm-160 nm for optimum resolution and brightness. Signal-to-Noise Ratio: Determines the visibility and clarity of specimen signals relative to the image background. Dynamic Range: Defines the dynamic range or number of gray levels that are distinguishable in the displayed image. 16 bit ADC gives Time Resolution: The sampling (frame) rate determines the ability to follow live specimen movement or rapid kinetic processes. Readout Rate: Acquisition speed in serial registry. 10MHz camera can take 30 msec images with 512*512 pixels. Faster readout increases the electronic noise. Region of Interest: Subarray image provides faster acquisition (whole rows are read regardless of the image size). Binning: Combining the pixels, improves time resolution with poorer spatial resolution.

14 State of the Art EMCCDs

15 CCD Camera Noise Sources Dark Current (per sec): Spontaneous generation of electrons due to thermal noise. Solution: Peltier cooling down the CCD chip to -80. Typical value is less than a photon per sec. Readout Rate (per image per pixel): Electronic noise in on-chip preamplifier during converting electrons to voltages. Increases by increasing the speed of acquisition. Solution: Electron Multiplication Gain. Pixelation Error: Nonuniformity in each pixel size (typically <10%), sensitivity and spacing. Solution: Scientific grade cameras $$$$ Photon Shot Noise: Fundamental limit of light collection. Poissonian error. No solution. Bright samples are photon shot noise limited, whereas single molecules or fast acquisition is readout noise limited.

16 EM-CCD Excess Noise Factors Due to the probabilistic nature of the impact ionization process, the uncertainty in the gain produces excess noise factor (typically 1.3). This factors directly effect photon shot noise limit. The clock pulses may produce a secondary electron even when no primary electron is present for transfer. Clock induced charge, dark related signal! Signal = (I.Q e.t) Noise = [(I.Q e.t. F 2 ) + (D. F 2 ) + (N r / M) 2 ] 1/2 where F represents the excess noise factor, D is the total dark signal, N(r) is the camera read noise, and M is the on-chip multiplication gain.

17 At low light levels, EM reduces the readout noise below 1 electron/pixel/frame, significantly enhancing SNR. At high light levels, EM increases photon shot noise, which may reduce the SNR.

18 EMCCD CCD ICCD Single Photon Sensitive Good resolution - Pixel limited Good dynamic range possible Fast or slow readout Flexible - Operate as EMCCD or CCD (gain can be turned off). Conventional CCD amplifier on some sensors No photocathode! Relatively affordable High and broad QE Good resolution pixel limited Good dynamic range possible No multiplication noise No photocathode! Greater choice of sensor formats available Single Photon Sensitive Nano and Picosecond time-resolved gating possible Fast or slow readout NIR photocathode options No nano or picosecond gating (microsecond gating available on some recent interline EMCCD sensors) Multiplication noise (effectively increases shot noise by x1.41) Read noise limited - not single photon sensitive Limited readout speed due to read noise restraints QE restricted by photocathode (<50% max) Poor dynamic range need to operate at high gains Cross-talk between channels of MCP increased point spread function Higher multiplication noise Artefacts, e.g. halo, chickenwire Expensive Damage susceptible longevity issues

19 Future: Scientific CMOS Cameras CMOS (complementary metal-oxide semiconductor) In a CCD device, the charge is actually transported across the chip and read at one corner of the array. An analog-to-digital converter turns each pixel's value into a digital value. In a CMOS sensor, each pixel has its own charge-to-voltage conversion, and the sensor includes amplifiers, noisecorrection, and digitization circuits, so that the chip outputs digital bits. Each pixel can be read individually. The individual amplifiers induce "fixed pattern noise" that arises from switching and sampling artifacts of individual pixel amplifiers. Because each pixel on a CMOS sensor has several transistors located next to it, the light sensitivity of a CMOS chip tends to be lower. Many of the photons hitting the chip hit the transistors instead of the photodiode. With each pixel doing its own conversion, uniformity is lower. CMOS sensors allow gain manipulation of individual photodiodes, region-of-interest read-out, high speed sampling, electronic shuttering and exposure control.

20 State of the Art CMOS Cameras Low cost Low readout rate No need for EM, hence no multiplication noise Small pixels Faster speeds Rolling shutter mode!

21 Back-illuminated scmos cameras will outperform EMCCDs at moderate and high light levels EM-CCDs will continue to dominate low-light level applications.

22 1. Histogram Stretching Digital Image Processing

23 2. ADJUSTING GAMMA TO CREATE EXPONENTIAL LUTs Displayed value = 255 (Data value - Min)/(Max - Min), linear LUT Displayed value = [(Normalized value - Min)/(Max - Min)] γ, exponential LUT With settings γ<1, low pixel values are boosted relative to high values and reduces the contrast between bright features and the darker background. A setting of 0.7 approximates the response of the eye, allowing the image to more closely resemble the view we perceive when looking in the microscope. Conversely, γ>1 depress dark and medium gray pixel values and increase the visibility and contrast of bright features.

24 Corr. Image = M. (Raw Dark) / (Flat Dark) 3. FLAT FIELD CORRECTION

25 4. IMAGE PROCESSING WITH FILTERS Filtering is used to sharpen or blur an image by convolution, by weighting intensity of neighboring pixels in the original image to compute new pixel values in a filtered image.

26 4. IMAGE PROCESSING WITH FILTERS Filtering is used to sharpen or blur an image by convolution, by weighting intensity of neighboring pixels in the original image to compute new pixel values in a filtered image. Low Pass High Pass

27 4. IMAGE PROCESSING WITH FILTERS Median Filtering is used to smooth highly noisy data.

28 5. UNSHARP MASKING Prepare a copy of the original and blur it with a conservative blurring filter. Images with fine details (high spatial frequencies) require more conservative blurring than images containing big blocky objects (low spatial frequencies). Subtract 50 95% of the amplitude of the blurred image from 100% of the original. The higher the percentage that is subtracted, the greater the sharpening effect. Using histogram stretching, adjust the brightness and contrast in the difference image.

29 6. FAST FOURIER TRANSFORM 1. FFT selectively diminishes or enhances low or high spatial frequencies (extended vs. fine detailed structures) in the object image. 2. Image is transformed into frequency domain through an FFT command, the information is represented in two plots (images): magnitudes and phases. In the magnitude image, frequency information is represented at different distances from the central point. Information from large structural features (low spatial frequencies) is found near the center of the image, and vice versa. The amplitude at each location in the magnitude plot is proportional to the amount of information at that frequency and orientation in the image. ADVANTAGES of FFT Removing noise that occurs at specific frequencies (electrical interference, raster scan lines) Enhancing or removing periodic structural features of the object Identifying spatial frequencies of defined structures in an image Determining the periodicity and/or orientation of indistinct features that are difficult to see in the object image Detecting optical aberrations such as astigmatism Applying convolution kernels to the magnitude plot for sharpening or blurring

30 FFT Filtered Image