Merging Bright Field and Fluorescent Images from CCD

Transcription

1 Merging Bright Field and Fluorescent Images from CCD This is to address the issue of merging a Bright Field (BF) image of an H&E staining with Fluorescent images from the same camera all taken at the same resolution. The goal of this is to visualize the overlay of the nuclei in the BF image with the three channel fluorescence data from the fluorescent labels. Image J will be used for all of the image processing. Each image from this data set is the same size and same resolution, eliminating the need for alignment of the images. However, each image, including the single channel fluorescent images, is taken in scanned RGB mode yielding RGB color images of the single channel fluorescent data. This introduces the problem that the pixels in the single fluorescent channel images are polluted by intensity values from the other base color values dependent on intensity of the pixel. This must be addressed before image calculation in order to avoid artifacts and false signals in the wrong channels due to intense valued pixels in the individual RGB fluorescent channels. Figure 1: Raw Image Data FLB, FLG, FLR, and BF each image is in 2776x2080 scanned RGB format from the Zeiss AxioCam HR.

2 Converting RGB Single Fluorescent Channel Images to Grayscale The color format for the single channel fluorescent images is not suitable for calculations. These images must be converted to grayscale images so that they can be recombined in pseudo color and contain no bleed over into other color values. Follow these steps in Image J to create grayscale versions of FLB, FLG and FLR. File > open > FLB Image > type > 16-bit *Now is also a good time to threshold the image. Image > adjust > brightness/contrast *Use the histogram and the image to zero the background and threshold intensity. File > save as > tiff > FLB_gray Repeat this process for FLG and FLR Now apply the pseudo color to these images, and possibly make an overlay of the 3 single channels of fluorescent data. To see an overlay of the 3 FL images: Image > color > RGB merge check box keep source image so you can use single channel FL images for other calculations. Select appropriate images for each color channel (images must be open) Next, pseudo color the single channel images for the overlays. This is done by performing the RGB merge command as done above, however selecting only one channel and none for the other channels. Image > color > RGB merge File > save as > FLB_pseudo Repeat this process for FLG and FLR.

3 Preparing the BF Image for Processing The BF image will be kept in color format initially, but it probably needs to be optimized. It is possible to optimize the brightness and contrast of the entire image or just to optimize each color individually depending on the quality of the color of the sample and the data collection. The details of each are described below. It is best to try variations to see what brings out the staining the best and drops out the background the best. Using the auto feature on each color of the RGB image and then adjusting the extremes just slightly usually works the best. Keep in mind, the image will be split, further thresholded and inversed later before performing the calculations. First the simple way: File > open > BF Image > adjust > brightness/contrast Image > adjust > color balance File > save as > BF_clradjust Merging Images Now it seems the original four images exist in slightly better versions. The FL images have been converted to grayscale, thresholded, and converted back to pseudo color to eliminate any contribution of pixel values in the other color channels. The BF image has been adjusted for brightness and contrast as well as the color balance to give the best representation or enhancement of the colors of interest. At this point, it is possible to simply add the images, however this is not the result that is desired for the final goal of seeing the blue BF H&E stained nuclei with respect to the fluorescent data. Here is the sequence to adding images with image J. Note that this is a mathematical addition of the pixel values in the image array. Images can also be averaged, subtracted, multiplied, max intensity selected, etc. using this same image calculation function. *images to add must be opened

4 Process > Image Calculator > Select BF_clradjust for first image Select operation > add Select FLG_pseudo for second image Figure 2: Result of BF_clradjust+FLG_pseudo and BF_clradjust+FLRpseudo. Overlay with FLB is not shown. The results of the simple overlay are somewhat disappointing. The overlay of the FL data onto the BF image has a very bright background due to the white background of the pre adjusted BF image which flushes out the fluorescence signal. The places that are darker in the BF image due to the blue nuclei staining are flushed out by the intensity of the fluorescence in those places. FLB is not shown as an overlay because the blues which would exist in both images would not be distinguishable. That can easily be remedied, however, by choosing a different pseudo color for the FLB if necessary. Inversion and Merging Images The way to remedy this situation is to think about the color qualities in the BF image and the FL images. Essentially, the FL images have high intensity values for their representation of their signals. The BF images have low intensity values for their representation of signal. One of the images has to be inverted in order for the overlay to actually make sense. To see what the FLB images looks like with the BF, we can invert the FLB image and overlay that with our BF image. The signal in the FLB will appear yellow in inversion with a white background. This is best done

5 with a minimum intensity calculation for the two images. Adding two bright images will saturate almost everything and lose most of the detail. The minimum intensities are the stronger signals in these images. File > open > FLB_pseudo Edit > inverse Image > adjust > brightness/contrast adjust the contrast of the image since it is an inversion now File > save as > FLB_pseudo_inverse Process > Image Calculator > select BF_clradjust as the first image Select min to calculate minimum intensities of the color channels in each pixel. select FLB_pseudo_inverse, then OK Save file Process > Subtract Background *Subtract background with rolling ball method to eliminate grayness, if helpful. Use Edit > Undo if effect is less desirable or to try with different settings. Figure 3: Result of Minimum Intensity projection of BF_clradjust & FLB_pseudo_inverse, and the same image with a Rolling Ball background subtractions with Rolling Ball factor set at 75.

6 It is also possible to inverse the BF image and merge it with the fluorescence images. Then, by adjusting the color balance to make the stained nuclei dominant in the image and thresholding to drop out the background, a representation of the blue staining of the nuclei is produced. The following step guide one through this process. Lots of time can be spent adjusting the thresholding at each step to yield the desired final product. With the BF_clradjust image open, perform the following: Edit > Invert Image > adjust > color balance adjust to optimize the positive signals in the red from the negative background Figure 4: BF_clradjust_inverse save BF_clradjust_inverse Image > color > RGB split select red channel from the RGB split Image > adjust > brightness/contrast *look at the BF_clradjust_inverse and threshold your red channel to levels that represent where your red positive signal is and that drop out the background. The more

7 thresholding is done, the more resolution is lost. However the trade off is being able to separate the positive signals out from the background. Image > Color > RGB Merge Select BF_clradjust_inverse for the channel desired for represented pseudo color (blue, to match its original stain, or red or green to recombine it with the FLB data). Select none for the other two channels, or select the images to combine it with. Figure 5a: Result of the blue stained nuclei with the FLG and the FLR images respectively. Figure 5b: The first image is a combination of the blue stained nuclei from the BF image merged with FLG and FLR together. The second image is a merge of FLG and FLB with their respective colors merged with the stained nuclei from the BF image represented in red. Simple BF Inversion FL Combination

8 The BF image does not have to be split in to its separate red green and blue channels. It is also possible to simply do the inversion, adjust brightness and contrast, adjust color balance and then add the desired channels together. The blue stained nuclei will appear red in the BF inversion, but the FLR image could be given another pseudo color if needed. The merge process of and inversed BF image combined with a fluorescent image is best done by thresholding the brightness of the inverted BF image to make it darker and to contrast out the red signal a bit and then using the addition function of the Image Calculator under the Process menu to add the pixel values of each image together. Figure 6: The BF_clradjust_inverse combined with the FLB and FLG images respectively using the addition function in the Image Calculator. Figure 7: The addition function combining the inverted BF image with the FLG and FLB.

9 You can also use the merge function under Image > Color > RGB merge, and merge the images together by selecting which channel you want them merged in. Then your BF image will only be present in that color channel (this is much easier than all of the thresholding and separation that was done before, however it is a slightly polluted signal because the intensities of the other two channels are combined into the color channel that you select as well. Figure 8: BF_clradjust_inverse merged with FLG, and a merge of FLR and FLB. Using a combination of the merge function, and the addition function, we can then take advantage of the blue nuclear staining in the BF image as being spatially separated from the FLB, and combine all of them together using the addition function to add the two images above, yielding a compilation of all 4 signals. Figure 9: FLB and BF_clradjust_inverse in blue added to FLG and FLR

10 However, it is also possible to simply add a merged image of the three fluorescent images with the inverted BF image using the addition function, which is a cute exercise though you can see the results are typically a bit ridiculous by the time you put it all together. Figure 10: FLB, FLG and FLR merged together in the first image, and the added to the BF image using the Image Calculator s addition function.