1.1 Background

DR is defined as the ratio of luminance of the lightest and darkest elements of a scene or image. In an absolute sense, this ratio can be considered the brightest to darkest pixel, but more commonly it is defined as the ratio of the largest nonsaturating signal to one standard deviation above the camera noise under dark conditions [10, 11]. An equation for DR_SNR can be described as the ratio of maximum to minimum input luminance level:

$\begin{array}{l}\hfill {\mathrm{DR}}_{\mathrm{SNR}}=\frac{L_{\mathrm{sat}}}{L_{\min }},\end{array}$

where L_sat is the maximum unclipped input luminance level, defined as 98% of its maximum value (250 of 255 for 24 bit color), and L_min is the minimum input luminance level for a specified SNR. The ratio can be specified as instantaneous for a single instant of time, or temporal, where the luminance values change over time, that is, light and dark elements over a series of images. Poynton [12] uses the definitions of simultaneous for the instantaneous case, and sequential for the temporal case. The primary focus of this chapter relates to simultaneous measurement, where a single frame of an HDR test chart from a video sequence is analyzed. New classes of imaging sensors are being developed that are capable of directly capturing larger DR on the scale of 5 orders of magnitude (5 log₁₀, 16.6 stops) [13, 14]. For comparison, human vision generally is agreed to have a simultaneous DR capability of approximately four orders of magnitude (4 log₁₀, 13.3 stops) [1, 15], and a sequential DR via adaption of approximately 10 orders of magnitude (10 log₁₀, 33.2 stops) [16, 17].

In order to objectively calculate the capture system DR, an understanding of the imaging and noise transfer function is required. An image sensor measures the radiant power (flux) Φ at each pixel j for each image i. The amount of photons $n_{p_j}$ that are absorbed at each pixel area A_j over time period t_i is given by:

$\begin{array}{l}\hfill n_{p_j}=\frac{A_j{u}_j{t}_i}{E_{\mathrm{p}}},\end{array}$

where μ_j is the mean incident spectral irradiance at pixel j. E_p is the photon energy defined by:

$\begin{array}{l}\hfill E_{\mathrm{p}}=\frac{hc}{\lambda },\end{array}$

with the wavelength of the photon given by λ, the speed of light c = 2.9979 × 10⁸ m/s, and Planck’s constant h = 6.626 × 10⁻³⁴ Js. Photons incident on the pixel over the exposure time generate an average number of electrons n_p based on the quantum efficiency of the sensor η(λ):

$\begin{array}{l}\hfill \eta (\lambda )=\frac{n_{\mathrm{p}}}{n_e},\end{array}$

producing a photocurrent i_ph and photon noise n_p at each pixel site. Thermally dependent leakage current in the electronics (free electrons in the silicon), known as dark current i_d contributes to the total charge, as shown in Fig. 1. In conventional current mode, the photocurrent produces a charge Q_j on a capacitive integrator generating a voltage V_j amplified with gain g. The voltage gain product V_jg includes readout noise n_r and amplifier noise n_a. Typically, a buffer circuit follows the gain amplifier contributing an additional buffer amplifier noise n_b, leading to an analog to digital converter (ADC) with quantization noise n_q, resulting in the measured pixel digital number (DN) d_ij. In linear systems, the DN is proportional to the number of electrons collected with an overall system gain G of units of DN/electrons.

Given a linear transfer model, the noise sources can be summed and we can write an equation for the total system noise at the output of the ADC as:

$\begin{array}{l}\hfill n_{\mathrm{total}}\left(j\right)=n_{\mathrm{p}}\left(j\right)+n_{\mathrm{d}}\left(j\right)+n_{\mathrm{r}}\left(j\right)+n_{\mathrm{a}}\left(j\right)+n_b(j)+n_q(j).\end{array}$

When specifying noise units, it is understood that the noise magnitude is the RMS value of the random process producing the noise [10]. If n_postgain is defined as the sum of the postgain amplifier noise sources n_r, n_a, n_b, and n_q, the variance is given as:

$\begin{array}{l}\hfill {\sigma }_{n_{\mathrm{total}}}^2={\sigma }_{n_{\mathrm{p}}}^2+{\sigma }_{n_{\mathrm{d}}}^2+{\sigma }_{n_{\mathrm{postgain}}}^2.\end{array}$

The variance represents the random noise power of the system, and the standard deviation characterizes the system RMS noise [18]. Finally, Reibel et al. [18] note that no system is strictly linear, and introduce a nonlinear contribution C_NL to describe the signal variance that is modeled by a dependence on the total electrons transferred to the sense node capacitor (photoelectrons and dark current electrons) and the overall system gain G resulting in:

$\begin{array}{l}\hfill {\sigma }_{n_{\mathrm{total}}}^2={\sigma }_{n_{\mathrm{p}}}^2+{\sigma }_{n_{\mathrm{d}}}^2+{\sigma }_{n_{\mathrm{postgain}}}^2+C_{\mathrm{NL}}.\end{array}$

A linear transfer model not only permits the summation of noise sources, it also ensures the relative brightness of the scene is maintained. To achieve linearity over the entire sensor, and to minimize fixed pattern noise (including streaking and banding), a radiometric camera calibration is generally required for each pixel, resulting in a total camera response function. The calibrated camera response function can be combined with optical characterization and other camera postprocessing functions such as gamma, tonal response, noise reduction, and sharpening to produce an overall camera opto-electronic conversion function (OECF) [19, 20].

Noise is meaningful in relation to a signal S (i.e., SNR) and can be defined differently depending on how the measurement is performed. If the measurement is concerned with the sensor itself, isolated from the camera postprocessing functions described above, noise can be referenced to the original scene. This can be accomplished by linearizing the data using the inverse OECF, and defining the signal as a pixel difference corresponding to a specified scene density range. Alternatively, if raw data is unavailable or the OECF is unknown (or beyond the scope or measurement capability), or if the focus is on complete system capability (optics, sensor, and camera performance) rather than sensor specific, the signal can be defined with respect to an individual patch pixel level. The evaluation of DR described in this chapter is based on the latter, with a goal of describing a straightforward method that can be used to produce an ever-increasing database of comparable camera system DR measurements. If the variance σ_signal² of the signal is known and the signal is zero mean, the SNR for the minimum input luminance can be expressed as:

$\begin{array}{l}\hfill {\mathrm{SNR}}_{L_{\min }}=\frac{{\sigma }_{\mathrm{signal}}^2}{{\sigma }_{n_{\mathrm{total}}^2}}\end{array}$

where ${\sigma }_{n_{\mathrm{total}}}^2$ is given by Eq. (6). Alternatively, Eq. (8) can be expressed as the ratio of the mean pixel value μ_mean to the standard deviation σ_{Std Dev}, and equivalently to the total RMS noise n_totalRMS(j) as:

$\begin{array}{l}\hfill {\mathrm{SNR}}_{L_{\min }}=\frac{{\mu }_{\mathrm{mean}}}{{\sigma }_{\mathrm{Std}\mathrm{Dev}}}=\frac{{\mu }_{\mathrm{mean}}(j)}{n_{{\mathrm{total}}_{\mathrm{RMS}}(j)}}.\end{array}$

Minimum input luminance level is often determined from the density step having a SNR of 1 or greater [11, 21]. The processing software utilized in this study named Imatest [22], performs automatic calculations for ${\mathrm{SNR}}_{L_{\min }}$ values of 1, 2, 4, and 10. It can be convenient to reevaluate Eq. (9) in terms of stops. Stops is a common photography term where one stop is equivalent to a halving or doubling of light, corresponding more closely to the relative luminance response of human vision [23]. We can express ${\mathrm{SNR}}_{L_{\min }}$ in terms of noise in stops by:

$\begin{array}{l}\hfill n_{{\mathrm{stops}}_{L_{\min }}}=\frac{1}{{\mathrm{SNR}}_{L_{\min }}},\end{array}$

where a ${\mathrm{SNR}}_{L_{\min }}$ of 1 is equivalent to 1 stop of noise. Fig. 2 illustrates the assignment of the arbitrary quality descriptions (high, medium-high, medium, and low) for 0.1, 0.25, 0.5, and 1 stop of noise as calculated and defined in Imatest, as well as the corresponding SNR values in terms of a luminance ratio as well as in decibels (dB). Finally, DR_SNR can also be expressed in terms of relative luminance or stops as:

$\begin{array}{l}\hfill {\mathrm{DR}}_{\mathrm{stops}}=\frac{{\log }_{10}\left(L_{\mathrm{sat}}\right)-{\log }_{10}\left(L_{{\min }_{\mathrm{SNR}}}\right)}{{\log }_{10}\left(2\right)}.\end{array}$

The question that remains is: How are the arbitrary quality descriptors chosen and why? We can start to answer this question by reviewing early 1940s era work in human signal detection by Albert Rose, known as the Rose model. The Rose model provided a good approximation of a Bayesian ideal observer, albeit for carefully and narrowly defined conditions [24]. Rose defined a constant k, to be determined experimentally, as the threshold SNR for reliable detection of a signal. The value of k was experimentally estimated by Rose to be in the region of 3–7, based on observations of photographic film and television pictures. Additional experiments using a light spot scanner arrangement resulted in an estimate of k of approximately 5, known as the Rose criterion, stating that a SNR of at least 5 (13.98 dB) is required to distinguish features at 100% certainty. The early empirical determinations of k had several issues. Dependency on a specification of percent chance of detection (i.e., 90%, 50%), included limitations of human vision, such as spatial integration and contrast sensitivity, and included dependence on viewing distance amongst other factors. The development of signal detection theory (SDT) and objective experimental techniques worked to overcome these limitations [25, 26]. For comparison, a binary (yes/no) signal known exactly (SKE) detection experiment, utilizing 50% probability of signal detection and an ideal observer false alarm rate of 1%, resulted in SNRs of 2.3 (7.23 dB) and 3.6 (11.13) for 50% and 90% true-positive rates, respectively [27].

More recently, the ISO 12232-2006 standard defines two “noise based speeds” for low noise exposures that is based on objective correlation to subjective judgements of the acceptability of various noise levels in exposure series images [28]. The two noise based speeds are defined as S_noise40 = 40, (32.04 dB) providing “excellent” quality images, and S_noise10 = 10, (20 dB providing “acceptable” quality images. Examining Fig. 2, a commonality point between the ISO 12232-2006 standard and Imatest occurs at the 20-dB point, pertaining to “acceptable” quality in the ISO standard and “high” quality in Imatest. We note that the ISO standard is based on prints at approximately 70 pixels/cm, at a standard viewing distance of 25 cm. For subjective models utilizing sensor pixels per centimeter values P differing from the ISO standard, an SNR scaling ratio of 70/P is recommended.

Previously reported results for objective calibrated DR test data have yielded limited results and the test methodology used is often unclear. The Cinema5D test website [29] notes “We have at one point in 2014 updated our dynamic range evaluation scale to better represent usable dynamic range among all tested cameras. This does not affect the relation of usable dynamic range between cameras,” however the author does not clearly state what evaluation scale is currently, or was previously, used. In another example, published on the DVInfo website [30], the author describes the minimum input luminance level as “typically, where you consider the noise amplitude to be significant compared to the amplitude of the darkest stop visible above the noise,” but does not further define the term “significant.” Results published on the DxOMark website [31] primarily include still camera and mobile phone tests, implementing a self-designed DR test chart that is limited to 4 density steps (13.3 stops).

2 Method and Materials

2.1 Design

Overall, the procedure for measuring system dynamic range involves four primary steps. Step one includes the setup of the test environment so that overall lighting conditions can be controlled, with reflected light minimized. The ability to set, maintain, and read room temperature is also of importance, as camera noise may be temperature-dependent. The second step is the setup and alignment of the camera under test (CUT) and HDR test chart in order to meet the pixels per patch test requirement. With setup activities complete, the third step includes the collection of sample image frames, followed by postprocessing to create compatible input frames for the analysis software. Finally, step four includes processing of the input frames through the analysis software and the review of data.

Three main components in the image capture pipeline that should be considered in measurement of DR include the optics, the sensor, and the camera processing workflow [32]. The optics are important in transforming the scene luminance to the image luminance at the sensor, and can reduce the DR as a result of diffraction, aberration, defocus, and flare [33]. The sensor can have limiting factors to DR as well, including saturation on the intensity high end, and noise limitations on the low end [2]. The camera data processing workflow begins with the color transformation (for single sensor systems), generally utilizing a Bayer color pattern followed by a luminance estimate as a weighted sum of the RGB values. Once the data is recorded, further processing may be required based on proprietary codes, log encoding, or other in-camera data manipulation. Care is taken to postprocess the collected data into a standard format for analysis. While the effects on DR of each of the three main components in the image capture pipeline can be analyzed in detail, a goal of this chapter is to focus on complete system capability, as may be used “out of the box.” The measurement and calculation, therefore, includes the combined effects of the optics, sensor, and camera processing as illustrated in the system level model of luminance conversion in Fig. 3.

2.1.1 Test Chart and Processing Software

A DSC Labs Xyla 21 test chart was procured that includes 20 stops of dynamic range via a 21-step, rear-lit, voltage-regulated light source. The Xyla chart, shown in Fig. 4, features a stepped xylophone shape, minimizing flare interference from brighter steps to darker steps. The Xyla chart includes an optional shutter system that allows for the isolation of individual steps in order to reduce the effect of stray light. The use of the shutter system comes at the expense of increased measurement and processing time, as each individual step must be imaged and processed, as opposed to the chart as a whole. The shutter system was not utilized in these experiments due to time constraints; however, the evaluation and characterization of the effect of stray light from the chart itself on the noise measurement is important future work. The Xyla chart is preferred to front-lit reflective charts, as front-lit charts are more difficult to relight evenly over time, as test configurations change or the test environment is altered. Rear-lit, grayscale, stand-alone films require the use of an illuminator, such as a light box, where special care is required to monitor the light source and voltage. The Xyla 21, being an all-enclosed calibrated single unit, simplifies test setup and measurement.

The Xyla-21 chart includes log₂ spaced steps, where each linear density step of 0.3 equates to one stop, a doubling (or halving) of luminance. The calibrated density data for the Xyla-21 chart used is provided in Table 1. The calibration is provided by the manufacturer, and the recommended upgrade (recalibration) date is printed on the chart.

Table 1

Xyla-21 density calibration data

Step #	1	2	3	4	5	6	7	8	9	10	11
Density	0	0.3	0.6	0.9	1.2	1.51	1.81	2.11	2.41	2.71	3.01
Step #	12	13	14	15	16	17	18	19	20	21
Density	3.31	3.61	3.91	4.21	4.52	4.82	5.12	5.42	5.72	6.02

The Stepchart module of the image analysis software Imatest [34] is utilized to process the captured images of the Xyla-21 test chart. The camera distance to the chart is adjusted to maintain approximately 50 pixels per patch horizontal resolution, required by Stepchart [22]. The Stepchart module includes a region of interest (ROI) selection tool that allows for the alignment and selection of ROI for zones as illustrated in Fig. 5. The light gray boxes are the selection region for each individual patch, with software controls allowing adjustment of all patches contained within the red outer box. The right image in Fig. 5 shows the selection tool with zoom and lighten functions turned on, which can be especially helpful when aligning darker zones. A useful alignment hint is to first align the ROIs based on a brightly lit captured frame of the test chart. Imatest will recall this alignment as the default for future images, therefore as long as the camera and chart are not moved, the brightly lit alignment can be used for test frames where the ambient lights have been switched off and darker zones are more difficult to align to. With zone regions selected, Imatest software calculates statistics for each ROI, including the average pixel level and variations via a second-order polynomial fit to the pixel levels inside each ROI. The second-order polynomial fit is subtracted from the pixel levels that will be used for calculating noise (standard deviation), removing the effects of nonuniform illumination appearing as gradual variations.

2.1.3 Test Procedure

The CUT is mounted on a tripod and placed in a dark test room, kept at an ambient temperature of 23± 2°C, with no light leakage along with the Xyla test chart. Camera lights and all reflective surfaces are masked to prevent reflections off the chart surface. A 25-mm Prime PL mount lens is attached to the CUT utilizing PL type mounts. For large format sensors, in order to maintain the 50 pixels per patch horizontal resolution, a longer focal length lens may be required, such as an 85 mm for the RED Dragon 6K. Cameras with nonPL mounts (such as C, Canon EF) utilize available compatible lenses.

A one-foot straight edge is placed across the front plane of the lens. Measurements are taken at each end of the straight edge to the face of the test chart to ensure the two planes are parallel to an eighth of an inch. The lens aperture is set to full open, and focus is set by temporarily placing a Siemens star on the face of the test chart. Once focus has been verified, the camera is set to the manufacturer recommended native ISO, and data collection begins. The ambient room lights are turned off and the door to the test room is closed, so that only the light from the test chart reaches the CUT. The lens remains at the lowest stop value (wide open) to ensure the luminance of at least the first 2–3 chips will saturate the image sensor at the slowest exposure time (generally 1/24th or 1/30th of a second). A few seconds of video are collected, and then the exposure time is reduced by one-half (one stop). Again a few seconds of video are collected, and the process is repeated for a total of at least 5 measurements to complete the sample collection process. Note that only a short video sequence is required, as only a single frame is used for the simultaneous calculation. Imatest does include a sequential (temporal) noise calculation, where additional frames could be utilized in the difference calculation, but the investigation of this will be future work.

Measured frames are processed through Imatest that can be configured to produce output figures relating to exposure, noise, density response, and/or SNR. A graph of normalized exposure pixel level (normalized to 0–255) is one output that is plotted against the step value of the Xyla21 chart. The original image data, whether 8- or 16-bit pixels, is first conver- ted to quad precision (floating point) and normalized to 1 before any processing is performed. The normalization to 255 is only for the purposes of the output graphical display (historically related to 8-bit file depth and the familiar 255 maximum pixel level). Therefore it is important to note that the full precision of the input file data is utilized in the calculations, independent of the normalized output scale. Sample normalized exposure data for the ARRI Alexa is shown in Fig. 6, noting that seven exposures were collected during this test. The collection of multiple measurements is to ensure an exposure was captured meeting the criteria of the maximum unclipped input luminance level L_sat defined in Imatest as 98% of its maximum value.

DR is calculated by Imatest, using the minimum luminance step, meeting the maximum RMS noise for each exposure time. This is shown in Fig. 7 for the ARRI Alexa. We can observe the exposure setting that resulted in the peak dynamic range was the one where the brightest chip was just saturated. In the case of the ARRI Alexa data, this was the exposure time setting of 1/190 of a second, resulting in 13.9 stops for a maximum 0.5 stop RMS noise utilized in defining the minimum luminance step.

The RED EPIC and DRAGON HDR modes are treated similarly, however, an estimate of the total dynamic range is made based on a combination of both the A and X frames. HDR exposure data for the RED EPIC is shown in Fig. 8, including a “shifted” X frame normalized to the sixth A frame step. Note that for both the RED EPIC and DRAGON, the longest exposure data was utilized to ensure saturation of the A frame, so that more of the X frame is utilized. There may be a limitation in the current setup, where the peak absolute luminance of the Xyla chart is not bright enough to fully extend into the X frame range.