Criterion (R1) vs. Asmik Ace (R2)

An MoC Labs DVD Comparison Review

For the purposes of this comparison, VOB files containing the first few minutes of the film were lifted off of both DVDs. The VOB files were then demultiplexed and losslessly exploded into individual 8-bit grayscale (256 grey levels) TIFF files using the Linux tools transcode and DeCSS which were adapted and compiled for Mac OS X by the author. We chose to use the first 2750 frames of the film for our comparison, shown in a large montage here. This constitutes the opening title sequence of the film. Our 2750-frame image sequence is defined in such a way that frame #2 is the very first frame of waving grass, while frame #1 is the preceding black frame. Our final frame, frame #2750, is the black frame appearing immediately after the final title card has faded out — this is also the very last frame of Chapter 1 of the Criterion disc, which has a chapter change at this point.

The Criterion and Asmik Ace DVDs are based upon source material originating from the same negative. The material is of such excellent quality that it is difficult to find a significant flaw within our 2750 frames with which to illustrate this fact. Frame #1301 of the film does however contain an easily spotted white speck, indicated below with a small arrow. The presence of a white blemish is indicative of dirt from a negative source (perhaps the original negative) and would be present on every single print made from that negative. It would, in principle, be difficult to determine whether or not the two transfers are based upon the exact same print, as Criterion claim to have performed in-house digital cleanup on their high-definition master. We are however still searching for evidence of such cleanup within our relatively small sample of only 2750 frames — both white and black scratches/specks appear to be identical between the two versions.

By studying the two frames displayed above, the following is immediately apparent: (a) Criterion has centered the film vertically, while Asmik Ace has the film sitting slightly too high up on the screen, (b) the Asmik Ace transfer includes thin borders on both sides of the image, making it more overscan-friendly than Criterion's, (c) Criterion's transfer is significantly cropped on all four sides, see for example the lower left hand corner of the image, and (d) the Criterion transfer appears (by subjective, visual inspection of the image as seen on our computer monitor) somewhat darker than the Asmik Ace transfer, as also pointed out in the DVDBeaver review. But let us have a closer, more objective look at things.

CROPPING
The top left hand corners of the above image pair are shown below in an enlarged view, demonstrating the severity of Criterion's cropping. Asmik Ace has provided "too much" of the film frame (by including a black overscan area on the sides), and Criterion has provided too little. Criterion has removed 9 lines from the top, 9 lines from the bottom, 21 pixels from the left hand side, and 16 pixels from the right hand side, thus leaving the viewer with a whooping 10% less frame real-estate than what is offered by the Asmik Ace disc.

Asmik Ace	Criterion

Criterion has undoubtedly tried to do us a favor by cutting away the "soft" edges of the film frame, providing a solid image with sharp edges. These cosmetically perfect edges are however only seen by those of us who have gone through the trouble of removing overscan. However, the reason for removing overscan in the first place is none other than to be able to see the entire film frame! By physically removing real-estate from the image, information is lost forever (as it were). Asmik Ace has retained the whole image (and then some), thus permitting those of us who wish to see the whole film to do so, and those who want sharper edges to apply their own matte, for example by using a software DVD player's built-in functionality (where available) to mask off any unwanted regions, or by carefully setting a hardware DVD player's incremental zoom.

IMAGE BRIGHTNESS AND CONTRAST
As mentioned in our introduction, the first and last frames of our two image sequences are "black." Just how black are they? An analysis of pixel values within these film frames (excluding the top and bottom black bars) reveals that the Criterion sits at solid black, with a pixel value of 0 (zero) at all pixels within the frame. The corresponding Asmik Ace frames have a mean pixel value of 19 (nineteen). As mentioned above, the grayscale used in this comparison is 8 bits, i.e., 256 levels (0–255). The Criterion black level is thus at the 0% level, while the Asmik Ace black level sits at 7.5% of full range (much too high). Black level is also referred to as pedestal or setup — more on that later.

Now let us define two Regions of Interest (ROI) in the Onibaba frame: one square (ROI A) and one circular (ROI B) area, positioned as follows:

Criterion and Asmik: Region of Interest A	Criterion and Asmik: Region of Interest B

We now take all 2750 Criterion frames and stack them like cards in a deck of cards. Now imagine shooting a bullet straight through the deck of cards, first through ROI A (a square bullet) and then through ROI B (a round bullet). The bullet, on its passage through the 2750 "cards," encounters various levels of image brightness. The following plots show this brightness variation. The average pixel value within ROI A as a function of time is shown on the left, and the average pixel value within ROI B as a function of time is shown on the right. Click plots to enlarge.

Criterion: average ROI A brightness vs. time	Criterion: average ROI B brightness vs. time

The sharp drop in brightness near the beginning corresponds to the moment the camera cuts away to inside the Hole, and both ROI A and B are in darkness (cf. the montage). The pixel value at the fairly flat bottom is 10 (ten). Corresponding plots for the Asmik Ace disc is shown in the following figure:

Asmik Ace: ROI A brightness variation with time	Asmik Ace: ROI B brightness variation with time

As before, the sharp drop in brightness near the beginning corresponds to the moment the camera finds itself inside the pit. The pixel value at the flat bottom is, as seen, 18 (eighteen) on our scale from 0 to 255. The Criterion signal thus clearly goes into deeper blacks than that of the Asmik Ace. By comparing the maximum signal values in the top two plots (227 and 239) with those of the bottom two plots (221 and 225), it becomes clear that Criterion (top) not only goes into deeper blacks, but also into whiter whites. Criterion is apparently making better use of the available dynamic range. Note for example the right hand side portion of the Criterion ROI B plot. It is the brightest part of the plot, at one point reaching as high as 239 (recall that this is average ROI intensity, individual pixels may be brighter than this) amidst frequent intensity modulations. Look closely at the top right hand corner (ROI B) of the image in the following movie, which includes frames #2594–2694 from the Criterion disc:

[ QuickTime | AVI ]

The movie shows that it is the waving grass that causes the observed intensity modulations. The brightest intensity is reached when we for a few brief moments have a clear view of the sky in ROI B. Criterion brings us the brightest skies, and is therefore not "darker than the Asmik Ace" (as is often claimed), after all.

Is the Asmik Ace transfer indeed "sharper" than the Criterion transfer, as has been claimed by some? We'll look at that next. The below is frame #535 from the Criterion and Asmik Ace disc, respectively. Click on frames to enlarge (note again the unfortunate cropping present in the Criterion, which here impacts the top of the frame composition). We have defined a small horizontal rectangular region within the image (yellow rectangle). Below the images are the respective horizontal (vertically integrated) intensity-variation plots for the area enclosed by the rectangle.

Criterion	Asmik Ace

Some quick calculations based on these plots tell us that Asmik Ace contrast is higher, thus the perceived enhanced "sharpness." This may be explained as follows: Asmik Ace loses much dynamic range due to the high black level pedestal (see the relatively bright "dark areas" of the image, as compared to Criterion). Since nineteen of the 256 grayscale levels are already taken, right off the bat, it is not surprising that the Asmik Ace has to swing through a greater intensity step in a shorter distance than the Criterion. It is like turning up the Contrast control on your TV: it may "look" better to the untrained eye, but it invariably results in loss of detail (somewhat analogous to employing an analog-to-digital converter with fewer bits); the Criterion transfer is therefore preferable, not only when it comes to brightness, but also when it comes to contrast.

Technically, when converting from YCrCb (as defined by the ITU-R BT.601 standard) to RGB, as we have done for the purposes of this review, luminance values will be found mainly in the 16 to 235 range. The following are the image histograms corresponding to one single frame taken from the sequence of waving grass, Criterion frame on the left and corresponding Asmik Ace frame on the right. (For those not familiar with image histograms: the abscissa shows intensity value (0–255) while the ordinate gives the frequency of occurrence within the single frame of each intensity value.) These histograms confirm that the intensity is generally concentrated within the 16 to 235 range — in this case with intensity concentrations more towards the dark (deep in the grass, left hand size of the horizontal axis) and the bright part of the intensity range (our sliver of sky, right hand side of the horizontal axis).

Criterion	Asmik Ace

Luminance values of less than 16 are referred to as super blacks (frequently used by Criterion, as we have seen), and values greater than 235 are super white. The 601 document referred to above states that the signal level is permitted to occasionally extend beyond level 235. Any use of intensity values in the range 235–255 range is likely to result in crushed whites (and inevitably loss of detail) when viewed on an NTSC TV (after 7.5 IRE setup has been added by the DVD player video output circuitry). Let us look at which of the two discs make most frequent use of super whites.

Recall our bullet analogy from above. We now abandon our ROIs and instead fire one bullet through each and every pixel of our image. The bullet, as it plunges through our deck of cards, is told to remember only the highest intensity it encountered during its journey through 2750 cards (corresponding to 91.7 seconds in time). Since we fired one bullet per pixel (a lot of bullets!), we can use this information to build up a complete image consisting of only the highest pixel intensity ever occurring at any given pixel location during the 91.7 second time slot. The resulting "maximum value" image is shown below, Criterion on the left and Asmik on the right.

Criterion	Asmik Ace

Displayed immediately below these images are their respective histograms, showing the statistical distribution of pixel values within the image. Clearly, Criterion make more frequent use of whites, and isn't scared of going into super whites. The saturation value (255) is actually the most frequently occurring single value within this composite image — the histogram is seen touching the vertical axis on the far right (it is hard to see, but note the "thicker" vertical axis). The Asmik Ace histogram shows that the Japanese have backed off considerably from using pixel values approaching the saturation point. This is normally a good thing — and something which Criterion should have done as well — but in the case of the Japanese disc with its high setup this only contributes towards further compressing their already restricted dynamic range.

Discussion and Conclusion

Notwithstanding severe cropping and crushed whites, as far as image quality is concerned the Criterion disc is the preferred disc. Its dynamic range is far superior to that of the Asmik Ace disc. The main problem with the latter disc is its high setup. It is exactly this high setup that is the real reason behind the erroneous statements seen in several fora to the effect that "the Criterion is darker than the Asmik Ace." Let us have a closer look at the issue of setup.

In Europe and Asia, analog video's blacks sit at zero voltage (0 mV PAL, 0 IRE NTSC). North American analog video has — for historical reasons having to do with imperfections in TV sets on which video was intended to be watched — its black level raised about 7.5% from the zero-voltage black level used elsewhere in the world. This is what is referred to as 7.5 IRE pedestal or setup. (Black level for NTSC was originally 0 IRE but had to be changed to 7.5 IRE when color was introduced. Eventually 7.5 IRE was no longer necessary, but the standard was never changed back to 0 IRE, with the exception of Japan which reverted to 0 IRE in the early 1990s.)

So, Japanese DVD players add no setup and American players add 7.5 IRE of setup. It is important to note that setup exists only in the analog world and has no relationship to DVD Video as an encoding scheme or as a disc format. No properly-recorded digital component format includes setup. Most DVDs are mastered, and indeed should be mastered, for 0 IRE. (Note that in our above comparison we ran transcode directly on the VOB files themselves, resulting in a truly objective picture of the facts, independent of any player-introduced setup or other idiosyncrasies).

In the case of the Asmik Ace Onibaba (and several other R2 Japan discs we have come across), the image — as digitally encoded on the disc — has for some odd reason had 7.5 IRE setup already added. This is not a good thing, as it is the responsibility of the analog output of the DVD player to add setup where appropriate. North American DVD players are, as already implied, adjusted from the factory to add 7.5 IRE on its analog output by default. When playing back the Asmik Ace disc on such a player, the "player setup" is thus added to the already "ingrained" setup, yielding a total of 15 IRE with a resulting lack of deep blacks and and limited dynamic range. Fortunately, some players have a menu option (or a small switch at the back of the unit) that will allow one to switch off setup on the analog output. This menu option (or switch) needs to be manually set to "0 IRE" or "no setup" when watching Asmik Ace's Onibaba. Don't forget to switch it back to its default setting (IRE 7.5) before popping in the Criterion disc (and most other DVDs, for that matter). As an aside: Even in the case of a DVD correctly mastered for 0 IRE, setting a DVD player to output 0 IRE can in some cases result in additional detail in the blacks. However, not every display device can handle 0 IRE properly, and those that can do require proper calibration — they should also have a 10-bit video processor.

Finally we note that it is only thanks to the existence of the DeCSS algorithm that we were able to perform an intelligent assessment of the quality of a product and thusly inform the potential customer before he/she parts with hard-earned money. Yet another reason why consumers should call for a ban on DVD encryption.

(This article has concerned itself exclusively with the video aspect of DVDs. The topic of audio has been treated elsewhere — see Unwanted Frequencies and PAL Speedup.)

Acknowledgments and References

Many thanks to Paul Whitelock and Jon Robertson for helpful comments. Paul also supplied half of the VOB files required for this comparison review. The transcode source code can be downloaded from here. For general image analysis, we use the platform independent ImageJ application.