How Many Ways Can You Count Your Ks?

Transcription

1 How Many Ways Can You Count Your Ks? Siegfried Heep Modern VideoFilm ASC / SMPTE Meeting, Welcome. This presentation skims over the tops of the trees, so to speak. It s kind of like: Oh look! There s a tree with round leaves, and there s one with a pointed top. It covers many topics in a condensed form, including pixel counts, film scanning, camera Bayer patterns and 4:2:2. Hopefully it will encourage discussion and some deeper explorations into these topics. >> Don t miss slides 7, 15 and 19. These notes and slides represent the fourth written version. Become a SMPTE member. 1

2 Topics to be Covered What is a Pixel? What is a K? DPX File Format Film Camera Apertures Projectable Image Area SDI - Serial Digital Interface Motion Picture Camera Imaging Sensors These are some topics that we am going to cover tonight. What is a Pixel? - covering Pixel Properties, and What is a K? Counting Pixels Then we ll review some SMPTE standards, like: DPX File Format Film Camera Apertures Projectable Image Area SDI and then I will go over several Motion Picture Camera Imaging Sensors, including Film and Electronic Cameras, focusing on the sampling structures and the focal plane. 2

3 What is a Pixel? Picture Element Pixel Counts Pixel Properties What is a pixel? It s a term for picture element. We count the pixels and we look at the Pixel properties 3

4 Pixel Count, What is a K? Kilo - metric modifier, one thousand units - kilogram, kilometer, kilobyte, K (pixels) Like kilobytes, pixels are binary addressed, usually, one K = 1024 instead of = 2K 4096 = 4K 8192 = 8K 1920 = 1.9K 7680 =? Binary addressed means that the locations of pixels are determined by binary counted numbers. Common K counts are 2K and 4K, as 2048 and Likewise, 1920 pixels would be like 1.9K. 4

5 2K or 4K Pixel Counts 2048 pixels wide 4096 pixels wide 3 vs. 12 Mega pixels (quadruple the amount) With square pixels, the width by the height is the Aspect ratio by 1556 = 1.32 to by 1080 = 1.78 to 1 In this example, the pixels are all identical. They are all RGB. The terminology of 2K and 4K comes from the number of pixels wide, 2048 and With square pixels, the width by the height determines the aspect ratio. For example, 2048 by 1556 would be 1.32 to 1. Likewise, 1920 by 1080 would be 1.78 to 1. In both of the these two examples, as film and HD, a photographed widescreen image would utilize these horizontal pixel counts with as many vertically counted pixels as required to fulfill the intended aspect ratio. 5

6 Components of color RGB, or YCrCb Pixel Properties Bit depth per component 8 bit, 10 bit, 12 bit, 16 bit Color Primaries and White Point Level Scaling (data range) Video Level Scaling (with data Headroom) at or Full Range (computer Graphics levels) Tonal curve (gamma) gamma, linear, log, printing density Components are color channels, like RGB, which is Red Green and Blue. Y Cr Cb is luminance with two color channels. The bit depth is what gives each color channel enough accuracy for smooth shading, so there is no visible contouring or gradient edge lines on smooth surfaces. And you also want to have enough bit depth to keep down the noise floor, avoid visible quantizing, and to have enough dynamic range. For what we do, 10 bit is probably the most common bit depth. The Color Primaries determine the range of colors that are possible, or the color gamut. But they don t just establish the boundary limits. They also define the colors throughout the range. In other words the color primaries are mixed in various amounts to make all the other colors. When all three primaries are Full On, you get the White Point color. The Level Scaling is a choice of Video Level Scaling versus Full Range. Video Level Scaling provides data Headroom for Y Cr Cb, and in a few cases, for RGB channels. Tonal Curve, usually called Gamma, affects the shade of colors and gray scale. It is used to take full advantage of the bit depth, based on the human eye s sensitivity to see differences in dark versus light. And, Printing Density is Gamma plus Log. On the next slide, let s take a look at the last two properties, the Gamma and Level Scaling. 6

7 Examples of changing settings low high clipped applied twice GAMMA CURVE VIDEO LEVEL SCALING Picture is derived from NIST test frame Here are some examples of how changing the Gamma Curve and Level Scaling would affect how the contrast of an image can look. The middle picture in each one of these examples is how it is supposed to be. I didn t pick it, but it s my job to make sure it doesn t change. If I don t select the right gamma setting, or choose the correct level range, this is what can happen. On the stairstep, notice that the GAMMA affects the mids more than the ends, and the LEVEL SCALING affects the ends more than the mids. To see the effect of GAMMA in the picture, take a look at her right cheek.to see the effect of LEVEL SCALING, take a look at the shadows on her neck and the highlights on her left cheekbone. If the RGB is not correctly identified as either being Full Range (Graphics levels) or Video Level Scaled (with data headroom), and then when Converting from one color environment to another, it s possible to make a mistake and end up with something like the examples you see here. To me, the one on the far right also looks a little softer and less sharp, only because it has less contrast. 7

8 Typical Pixel Properties HD D-Cinema In this table, the parameters of the DCDM for D-Cinema, and Rec. 709 YCrCb for HD, are fairly well identified because they are distribution formats. Rec. 709 is for HD broadcast and home video. The DCDM is for D-Cinema theatrical distribution. Cineon is widely used for DI s and VFX, in the DPX and Cineon file formats. Cineon s colors and printing density are based on film as it comes back from the laboratory, so there are no hard and fast numbers. Interestingly, the gamma on film and in Cineon is slightly different for each color component. The quasi-log of Panalog bends at the ends and has a data range of Generic RGB just needs to be identified all around. Either that or just try some settings and you get what you get. Many times it just depends on how a monitor was calibrated. Rec 709 YCrCb is always video level scaled. RGB is normally Full range, although sometimes it s video level scaled, especially for Rec 709 RGB. The XYZ primaries are specified by the CIE. XYZ is kind of like RGB on steroids. E is a specific white point. It is Equal Energy, where all three of the CIE XYZ primaries are at unity. DCI P3 is consistent. However, with the possible addition of a white point selection, (probably for a good reason) the white point would need to be further identified. The point here is that you have to know what data you have, and what the equipment is expecting. Otherwise, you can get unintended results. These Pixel Properties are independent of image resolution and aspect ratio. 8

9 Clear the palate with some nice fresh sorbet. We just covered a lot. We said that we count pixels, and that gives us the structure of the image. We also talked about Pixel Properties and how they affect the way the image looks. As for the sorbet, tonight I can only serve your imagination. 9

10 DPX File Format, Digital Frame SMPTE Standard 268 for storing digital images widely used for Digital Intermediate and Visual FX log - actually printing density, like Cineon lin - actually a gamma setting A bunch of pixels 10 bit Integer numbers (Code Values ) SMPTE is a collaboration of the whole industry. In the engineering committees, standards are written for consistent interchange of sound and picture. This standard for the DPX file format is used for digital frames, like Cineon film scans and digital camera images. It is widely used for DI and VFX. It s really just a bunch of pixels. It s a way to store images and transport them as data files. It uses Integer numbers, usually 10 bit. For example, these numbers (also called Code Values) can represent so many levels of density on a negative. There are other file formats as well, some of which also use Integer numbers, like TIFF, while others use floating point numbers, like Open EXR. 10

11 Film Camera Apertures SMPTE Standard 59 physical dimensions on film width A =.866 inches Style A: with sound (academy aperture) Style B: with sound (anamorphic) width A =.981 inches Style C: no sound (full ap) This SMPTE standard for Film Camera Apertures specifies the physical dimensions on Film. It has nothing to do with pixels at this point. Styles A and B have an area for a Sound track. The sound track area is used for theatrical prints. These drawings are shown as looking back toward the lens through the film. Normally, a film frame, when it s Displayed, has the sound track out on the left side of the picture. Style C is the full camera aperture. 11

12 Negative Film Scanning Cineon (Kodak) Quarter resolution - 2K context width A =.866 inches Physical dimensions on film equal pixel counts in digital 1 pixel = inches 220 pixels = sound track area width A =.981 inches 1828 pixels =.864 inches 2048 pixels =.968 inches 1920 pixels = HD A general rule is to scan the whole thing. That way the whole thing can be recorded back to film and inter-cut with the Original Camera Negative. For full camera aperture, 2048 pixels doesn t quite make it to.981 inches, so the side edges of a digitally scanned full camera aperture film frame are generally clean. Even though the radius corners can show up in the scan, they are usually outside the aspect ratio. Sometimes, the metal edge of a camera aperture plate shows up on one side of the scanned image. But that edge probably won t be visible on a film screen, as we will see on the next slide. When doing a project, choose one of these pixel counts and then size it all appropriately when recording the image onto film for theatrical projection, as either academy or anamorphic. The sizing can be done either optically or digitally in a DI. If you are dealing with mixed sources in a DI, in digital post production, you would have to choose one size, like one of these three pixel counts, and then re-size all other sources to that pixel count. That way you can combine the sources and cut them together. [ ] EXTRA 1536 pixels for the height was not quite enough, especially for anamorphic. So they made it 1556 pixels. height: 1536 pixels =.726 inches 1556 pixels =.735 inches 12

13 Projectable Image Area SMPTE Standard 195 width A =.825 inches usually matches the Ground Glass lines Image area is intended for projection Rest of film is intended for NOT projection 1746 pixels =.825 inches 1956 pixels =.924 inches The actual film image area that is projected may be smaller... keystoning (see note 2 in SMPTE 195) This image area is implemented in theaters with a projection aperture plate in the projector and masking curtains on the screen. These picture edge lines don t actually exist on the film, other than maybe on a photographed framing chart. It is usually made to match a Camera viewfinder s Ground Glass lines. This is the intended image area for projection. The wording on the slide is deliberate. The intention exists for both projection and NOT projection. For example, the intention is to NOT project sprocket holes, the sound track area, or any remaining image on film that is outside the intended projectable area, and outside the camera viewfinder s Ground Glass lines. The point is that this is the opposite approach of a safe action area, where a certain percentage of the whole displayed image is intended to be visible on the screen. In the 2K context, these pixel counts are for academy and for full ap. If we take the specified projectable area and do a reverse extraction, we get a full ap projectable area with 1956 pixels. In a nutshell, the 19 hundred and 56 pixels is the viewable image area for DI in 2K. However, the viewable area for a 2K D-Cinema DCDM is 2048 pixels wide. Therefore, in the D-Cinema mastering process, either a little more of the film image is viewed, or the image is sized up to fill 2048 pixels. So, this was counting pixels in file formats and film scans. 13

14 SDI - Serial Digital Interface SMPTE Standards 292 and 372 for connecting equipment 4:2:2 is single link Gigabit / sec. 4:4:4 is dual link - 3 Gigabit / sec. HSDL is at half frame rate (not SMPTE) A stream of pixels 10 bit Integer numbers (Code Values ) Now we are getting into serial interfaces, and more Pixel Properties. SDI is widely used for connecting equipment for display and processing. It uses coax cable with 75 ohm BNC connectors. Use dual link for full bandwidth 4:4:4. High Speed Data Link is not a SMPTE standard, but it is used to transport images with a pixel count of 2048 by 1556, like a film scan. This serial interface is 10 bit integer just like the DPX file format. However, the very highest and lowest Integer numbers are reserved for synchronizing marks. 14

15 4:4:4 or 4:2:2 4:4:4 is RGB or YCrCb YCrCb is an intermediate format to and from RGB Y is luminance, a monochrome gray-scale Cr and Cb are rectangular coordinates that identify the color 4:4:4 YCrCb 4:2:2 is always YCrCb 4:2:2, some of the pixels have no chroma Chroma is sub-sampled 4:2:2 interface 4:4:4 can be either RGB or YCrCb. In this example, it is shown as YCrCb. Y Cr Cb is luminance, with two color channels. Luminance is a monochrome gray scale, made by adding together red, green and blue. CrCb is a color description, identifying the color that belongs with the shade of luminance. The color is identified as two rectangular coordinates. Rectangular coordinates are defined by going so far horizontally and so far vertically to identify a point within a rectangle. In this case, the rectangle is surrounding a color wheel, so the points within the rectangle define specific colors. Images start out as RGB, and get displayed as RGB. To go from RGB to YCrCb and back again is done with a set of mathematics. It s part of Rec. 709, and it s consistent. Basically, the RGB primaries are matrixed into YCrCb. YCrCb is really an intermediate format. The conversion from RGB to 4:4:4 YCrCb is done with each individual pixel. Then, a conversion from 4:4:4 YCrCb to 4:2:2 is done by filtering the chroma to half horizontal resolution, resulting in sub-sampled chroma. The chroma only has half the number of pixels. With 4:2:2, every other pixel has only one component instead of three, with 1/3 less data. 4:2:2 is always YCrCb. We can get away with this because, at the right distance, the human eye sees more detail in luminance than in color. Going back to RGB first requires a restoration of full-bandwidth chroma. The missing chroma pixels are interpolated, or estimated, by averaging the adjacent chroma pixels. When the pixels are RGB, every pixel has three components. 15

16 Motion Picture Camera Imaging Sensors Film 3-chip Bayer - mosaic Color Striped Foveon Getting into more pixel manipulation, lets look at some Motion Picture Camera Imaging Sensors. 16

17 Film Randomly located photon sensors (film grain, dye clouds) Focal plane established for commonly used lenses Layers Co-sited colors Pictures are from Kodak publications H-1 and H-188 When color film gets exposed to light, photons strike the Silver Halide crystals in the film emulsion and some valence electrons get knocked free. When the exposed film gets processed, the Silver Halide crystals with enough missing electrons become developed Silver crystals and activate dye couplers in their surrounding areas. In the middle drawing, you can see the dye-clouds forming around the developed Silver crystals. When the film processing is complete, the silver has been washed away by the fixer (bleach), and the dye clouds remain. Doing a bleach bypass leaves the silver on the film. In the bottom picture, the dyes in these layers have been activated by processing the film after it was exposed to full spectrum light. These Blue, Green and Red light records are stacked as Yellow, Magenta and Cyan dye-forming layers. It is interesting to note that when film gets scanned to digital, it s sampling the randomly located film grain, which is actually Preserving the image sensor structure. 17

18 3-Chip Uniformly positioned photon sensors Focal Plane different than Film - therefore different lenses Prism splits the light into colors 3 paths onto 3 chips Co-sited colors TRICHROIC PRISM ASSEMBLY Picture is derived from Three chip cameras use a prism to split the light into three color spectrums, for three imaging chips. The chips are registered so that the colors line up with each other, so the colors are co-sited in the camera. The Focal Plane is based on the physical distance from the lens to the sensor. With a 3-chip camera, you have to go through all this prism stuff to get to the sensor, so the sensors cannot be placed at the same Focal Plane as Film. Therefore, different lenses have to be used. In 3 chip cameras for Cinema, a typical chip size is 2/3 of an inch. Some examples of cameras that use these chips would be the Thomson / Grass Valley Viper and the Sony F

19 Bayer Pattern Mosaic pattern of single color photo-sites 4K? Over-sampled Single chip Focal Plane can be the same as Film - use the same lenses Not Co-sited colors Bayer Post Processing to RGB (de-mosaic) RAW IMAGE SENSOR PATTERN AFTER PROCESSING The Bayer Pattern is probably the most widely used type of imaging chip for digital cameras, including consumer still-image cameras. Here we have an example of the Mosaic tiled pattern of single color pixels (photosites) from a Bayer pattern imaging sensor, and an example of post- processed, full color RGB pixels. In the RGB pixels, the upper left one is showing the four Bayer pattern sensors to enhance the relationship between them. One way to de-mosaic from the raw sensor pattern is to interpolate the red and blue colors that are missing for each green photo-site. Every green photo-site is surrounded by 2 red sites and 2 blue sites. In this simplified example, the missing red color could be interpolated, or estimated, by averaging the 2 adjacent red photo-sites. Likewise for the blue color. With this kind of processing, the three RGB color components of the processed pixel would all represent the same virtual photo-site. Unlike for film grain, we do not want to preserve the tile structure of this imaging sensor structure, but process it out. The quality of that processing is one of the most important things for a Bayer Pattern camera. This chip is similar in size to 35 mm film. Some examples of cameras that use this chip would be the DALSA and the ARRI. 19

20 Color-striped Striped pattern of single color photo-sites 6K? Over-sampled Single chip Focal Plane can be the same as Film - use the same lenses Not Co-sited colors Processing to RGB RAW IMAGE SENSOR PATTERN AFTER PROCESSING This type of sensor is similar to the single chip Bayer pattern sensor, but this chip has a different pattern. Both the Bayer and color-striped one chip camera sensors have potential for edge artifacts, because the colors are not co-sited, or in the same location. The image quality is very dependent on good quality Processing to RGB. An example of a camera that uses this chip would be the Panavision Genesis. 20

21 Foveon Uniform pattern of photo-sites Placed at different layer depths Single chip Focal Plane could be the same as Film Use the same lenses Layers - Colors filtered in the silicon Co-sited colors Not currently used for motion picture cameras - (yet?) There is a single chip camera sensor that does have co-sited colors. The Foveon chip is not currently used for motion picture cameras. Reportedly, a few still cameras use a Foveon Image Sensor. Filtering colors with silicon may be technically interesting in getting the desired colors. 21

22 Thank You If the participant becomes aware of the technology, the dream is shattered. unknown Thank you to Michael Will at Modern VideoFilm for helping with the pictures. If the participant becomes aware of the technology, the dream is shattered. Thank you. 22