Leica R9/DMR test report: a critical appraisal of a first step to the future (27 August, 2005)
Introduction and background
The original R8 was introduced in 1996 after a five-year period of development. Around 1990 the photographic world looked quite stable and had produced a mature technology where electronic engineering and automation had taken over from the mechanical engineering and the craft of the individual photographer. Leica in this stage of evolution decided to opt for the time-honoured aspects of photographic craft and the mechanical excellence they were renowned for. And, to be honest, Leica would certainly have lost a head-on competition with the brutally efficient Japanese companies.
The choice to stay in a niche market at that time was a sensible one. At the same time, Canon was preparing for a major switch in photographic equipment and technique as the digital revolution achieved momentum. The Leitz family has always disliked the SLR concept because it was a direct challenge to the rangefinder philosophy. And it is fair to note that Leitz and later Leica reluctantly adopted the view that the RF-concept would be the exemption and not the rule in the photographic world.
The R8 then was the last attempt by Leica to try to develop an SLR system which emulated the M-concept as close as possible, by translating the specific SLR properties into an integrated whole of high quality engineering. Around the year 2000, it became clear that the digital transformation of photography waa a fact of life. The Leica company responded to the new challenge with the same attitude that they had adopted when the SLR challenged the RF concept in the early sixties of the previous century. The challenge would be only temporary and true photographers would stay faithful to the better instrument: the rangefinder. It did not occur and the Nikon F became the rule of things.
Around 2000 Leica Solms assumed that the digital revolution would transform the photographic world, but the real cognoscenti would recognize the virtues of film-based imagery and this would be a broad enough base for Leica to prosper in a niche. The development of the R8 had consumed all financial resources, and a totally new digital camera was not an option. The idea of an interchangeable back was not a novel proposition. The Zeiss Contarex had shown the way and several medium format cameras did prove the validity of the concept. At that time the M-line was booming and the interest in the SLR-market was quite limited. An early prototype of an interchangeable digital back for the R-body proved to be very expensive to manufacture and the initiative did not get a follow-up. But with the digital wave steamrolling over the filmbased photographic culture, it became clear that Leica could not stay in the fortress 'M' forever.
The original proposition was dusted off, but now the help of others was required. Kodak provided the sensor and Imacon the software. There is some validity in the question why Leica did not opt for a totally new digital SLR. The main argument is one of compatibility: Leica has always stressed the non-obsolescence rule: a new 2005 lens should fit the earliest 1954 M body. Any new digital SLR body must accept the current R-line of lenses. A digital back seemed the best option to satisfy the many R users who yearn for a digital solution.
The choice for a digital back does define the main buyer population: the established R8 and R9 users.
The R9/DMR combo
The specifications are now well-known within the Leica community and do not need to be rehearsed. What has to be discussed however is the hybrid character of the combination. The DMR arrives with some tooling to remove the standard film back and to fit the DMR to the R9 chassis. This is a simple job and the DMR is fitted to the body in seconds. Leica has spent some effort to argue for the virtues of this hybrid design that allows the user to choose between silver and silicon. In my view this approach lacks substance. When the choice has been made to go for the digital workflow, there is little incentive to stay loyal to the film-based photographic culture. It is quite true that silicon-based photography and silver-based photography are related as synthetic music to live music, but most people choose with their feet and a recent survey indicates that from all photographers only 5% will hold on to the silver halide emulsions. At least for the time being!
And if I would adopt a dual strategy, the Leica product is very expensive. I have to pay up to Euro 7500.00 for a DMR/R9 combo and for the same amount of money I can buy a Nikon F6 and a Nikon D2X. Or I can choose a Canon D5 and an EOS 3 if I want to use the same lenses with the same sensor area. The DMR then is primarily a product for current Leica users who do wish to continue to invest in the Leica lens line or have done so in the past and do not want to write off their investment in one stroke.
The hybrid character can be experienced when one notes that the mechanical (front end) R9 part and the electronic (back end) DMR part are not integrated from a user perspective. The engineering quality of the DMR is great and the quality of the finish is up to the high Leica standards. The information that is displayed on the DMR screen is not available on the R9 viewfinder and visa versa. This is not a big problem, but one is constantly reminded to the dual nature of the solution.
Compared to the Nikon D2X, the DMR shows some weaknesses: the Leica finder is not the brightest one can buy, the DMR LCD display is quite crisp, but not large, the zoom factor is limited to 4x (the D2X has a 10x zoomfactor, which allows for a better analysis of the image), the Nikon has a buffer for more than 20 images, the Leica has 9 images at a lower motordrive speed (2 frames per second versus 6 frames/second), the Leica battery is drawn empty quite quickly. Manual focussing is slow and to be very accurate one has to be quite concentrated.
The R9/DMR then is meant for the photographer who accepts and adopts a photographic style that is close to the slow-food movement that is emerging from the North-Italian eating culture: enjoy the pure food and take your time to taste it. And wants the convenience of digital recording and mathematical manipulation of the pixels. The fine art photographer, the studio worker, the wedding photographer and the portrait artist could benefit from this Leica concept, as the camera and back require a certain discipline and attitude to extract the best from it.
The image quality
Leica has stressed the fact that their solution has two main advantages: the optical quality of the Leica R lenses can be fully employed and the sensor size and construction (no low-pass filter) supports the recording of really fine detail. This test will focus on these issues. For this test I selected a Nikon D2x, a Canon 20D and an Olympus E1 as comparison systems.
A meaningful test of a digital system camera can adopt two strategies: one is to compare pure pixel performance and one is to compare practical performance. If you wish to compare the performance of the system, you should compare apples to apples, not oranges. The Olympus E1 has a smaller sensor area than all of the others.
A comparison test on a pixel for pixel basis implies that one should frame the subject in such a way that for every camera the same sensor area would have to be used or what amounts to the same: a smaller magnification factor should be used. To be specific: the sensor area of the DMR is twice as large as that of the E1. The magnification ratio must be twice as large for the E1 to be comparable. If I shoot a full frame portrait with the E, I should take the same picture at a wider distance with the DMR to compensate for the larger number of pixels. But this is not what you do in real life: you will shoot the portrait in both cases at such a distance that the face fills the frame.
This is not different from the well-known discussion about the virtues of the medium format negative compared to the 35mm negative format: you use a larger recording area to capture the same amount of information. That is how I proceeded in this comparison test: all scenes were shot at a magnification ratio that allowed the scene to be recorded over the full area of the sensor. The E1 is obviously at a disadvantage here: the test chart has to be recorded with 2560 pixels where the DMR can use 3872 pixels for the same level of fine detail. The size of the pixel is the same in both cases: 6.8 micron. The DMR then has 1.5 times the amount of pixels to record the same amount of information.
The vital statistics for the four systems are
E1: 2560 x 1920 pixels, 17.1 mm x 13 mm, 6.8 micron, 2.0x
20D: 3504 x 2336 pixels, 22.5 x 15 mm , 6.4 micron, 1.6x
DMR: 3872 x 2576 pixels, 26.4 x 17.6 mm, 6.8 micron, 1.4x
D2X: 4244 x 2848 pixels, 23.7 x 15.7mm, 5.5 micron, 1.5x
The testchart I use has the classical barline pattern, where the number indicates the actual number of linepairs per mm on the chart itself. The magnification factor determines the real linepairs/mm on the negative or sensor. This test shows clearly that the D2X is the winner with a crisp definition at the 2 lp/mm level. The DMR records at the level of 1.6 lp/mm. The 20D records at a level of 1.4 and the E1 at a level of 1.25 lp/mm. In every case one is inclined to include the next higher level, but that is immaterial for the ranking order.
Top row from left to right: Nikon D2X and Leica DMR
Bottom row from left to right: Canon 20D and Olympus E1
If we adjust for the sensor size, we may note that now the 20D surpasses the D2X and the E1 approaches the DMR in recording power. These findings may come as a surprise to some, but in fact the classical rules of small-format photography still hold in the digital age. If you maximize your negative area, you have a clear advantage over the person with a nominally bigger area, who is more wasteful with his coverage. And it also illustrates the other rule: a bigger negative area and a medium quality lens can compensate for a better lens and a smaller area.
The famous Geoffrey Crawley once did a test with a cheap Chinese medium format twin-lens reflex and a high grade 35mm reflex with a top class lens and had to conclude that the Chinese camera produced the better final image at the same size of print!
From left to right: EOS 20D and Olympus E1
These results also suggest that a comparison between digital cameras can be easily misleading if you do not state your test conditions in some detail.
It is quite interesting to note that the quality of the optics is less influential than is often assumed. The E1 has the highly acclaimed D-Zuiko 14-54mm and the Leica the excellent new Summilux-R 50mm. The Canon and the Nikon were fitted with bread-and-butter optics, the Nikon 28-105mm and the Canon 17-85mm. All lenses were stopped down to 8 and 11 to compensate for small focusing errors and optical deviations. The tests were repeated over three days with two different persons, so I had six different series of pictures in identical situations (studio flash and tripod and same subjects all at full frame magnification). The results presented here are the best ones selected from different series. It is also honest to declare that there was a wide variation in the results.
There were series with the DMR that are below what you get with the E1. It is quite intriguing to note that the variation is wider in the case of the E1 and the DMR, where the D2X and the 20D were quite consistent in their results. As the DMR and the E1 use the same Kodak chip technology, it is tempting to conclude that at cut-off frequencies the margin of error is smaller for the Kodak solution. The numbers shown in the results are the actual numbers on the testchart: when adjusted to the magnification we are talking here about high frequencies in the region of 40 to 60 lp/mm.
This is below the performance that can be attained with film, where I can record up to 100 lp/mm and more. But this is no longer an issue: silicon has won the match.
Below I show the MTF analysis from the four systems analysed: the E1 lens is clearly the best one and this one screams for more pixels to show its real quality. The other three operate in the same league, quite good for the situation, but one sees also that the Leica lens cannot play out its high quality. The lack of the low-pass filter does not translate into better performance. On the contrary: the study of the line pattern beyond the resolved ones, shows a scrambling of the high frequencies where the software tries to make sense of the information presented to the sensor. It seems that the Imacon software cannot cope with the high frequencies that are not blocked by the customary low-pass filters. The upshot is that the DMR has a more aggressive sharpening algorithm than the others. This can be clearly seen in the following pictures where the reflections show more aliasing effects in the DMR case.
The maximum resolution for the four systems can be calculated form the pixel size and is 90 lp/mm for the D2X, 80 lp/mm for the 20D and, 75 lp/mm for the DMR and the E1. The factual resolution, as seen on the testchart resolution is about 60 lp/mm for the D2X, about 50 lp/mm for the DMR, 45 lp/mm for the 20D and 35 lp/mm for the E1. The lower figures for the E1 are related to the size of the sensor area and the magnification ratio (distance and focal length).
At closer distances the E1 operates close to the theoretical optimum. Remember to read these figures with the 15% test margin I discussed in a previous paper.
Colour reproduction
All cameras were set to the automatic white balance option. In this case the cameras do perform over a wide range, but given the wide variety in personal tastes, I do not rate this performance. Note that all pictures in this report are Raw images, recorded in the camera with all settings set to 'standard' and developed with the Rawshooter program, again at standard settings. Differences in colour and contrast are differences in the way the software engineers assume that the 'negatives' should be processed by the software.
ConclusionA number of current convictions need to be restated.
In the silverhalide era the image quality at the print level is the combined result of the quality of the emulsion, the size of the negative, the quality of the lens and the quality of the developer.
This equation still holds in the solid-state era of photography: the sensor quality, the sensor size, the quality of the lens and the performance of the software (in-camera sharpening and out-of-camera RAW developers) determine the final print result.
In the silverhalide imaging chain the quality of the lens is the limiting factor in combination with the negative area: the quality of the emulsion is extremely high and on a level above all the other factors. Knowing the optical performance of a lens, you can safely predict the character of the final image.
In the solid-state imaging chain it is the software that determines the outcome. Theoretically the image capture size is of less importance as the information recorded on the small pixels can be transformed loss-free to a large format print. The technology behind the four-third format of the Olympus E1 assumes that this is indeed the case: optimized optics and very small pixel areas are the film-based equivalent of the microfilm with high-grade optics. It does work as can be seen from the resolution figures, but there is an imaging threshold where the combination of limiting frequency and sensor area start to fail. The 5 MP of the E1 do on excellent job on A4 format and a very fine job on A3 format, but as soon as we have more information elements in the subject than can be translated one to one on the sensor area, we cannot capture all the information of the scene. The E1 then should be operated as we are accustomed to do with the Minox or the 35mm (Leica) format: go to the subject as close as possible, do not waste precious negative area and be moderate with enlargements.
The 20D is fitted with a lens that is claimed to be a dedicated design for the small sensor area and the pixel size. This size is slightly smaller than that of the E1 and the lens is definitely not as good as that of the E1. But the larger capture area allows finer details to be recorded and the final result is simply better. Let us be real here: the 20D can resolve around 40 to 50 lp/mm and get this to the final print without any loss, thanks to the mathematical algorithms that incidentally were developed in the 18th century. The final print quality exceeds the image quality you can achieve with 35mm negatives and chemical prints, at least as we limit ourselves to the criteria of subjective sharpness and objective definition.
We should be careful here in our appraisal: in most situations 20 to 30 lp/mm are all we need for high quality fine-art and publishing work. All four cameras offer sufficient performance to reach this goal. A digital photographer today needs to have more expertise in the software with which to develop and manipulate the RAW images than in the past, when photographic experience was the core of the business.
The DMR performs above the level of the 20D, but the gain is less impressive than theory predicts and some magazines claim. The omission of the low-pass filter should allow for a higher capture quality, as the optical losses induced by the filter do not occur. Looking at the test chart results and the practical images, we do note a quite high resolution, but at the same time a very aggressive internal sharpening mechanism that introduces anti-aliasing not seen in the other three cameras. It seems illogical, to quote Spock, to destroy an inherently finer resolution with an aggressive sharpness algorithm.
The best results, almost on medium format level are delivered by the D2X, a clear indication that a not-so-impressive lens in combination with expertly developed software and a high number of pixels does the job. But on all counts as the colour chart shows: here Nikon is on fourth rank.
Final word
With the DMR, the Leica R user can produce results that are better than what we can achieve with the current standard 8 Mp class, but the DMR is not in the same league as the state-of-the-art top performers. That is partly the result of the long development cycle (two years ago the digital world looked different) and partly the result of the Imacon software. Actual results may be improved when the resolution champions of the Leica line (the 100mm, the 180mm and the 280mm) are deployed, but we should accept that one wishes to use all of the lenses in the R range. In the film-based world, Leica users automatically and with reason assume that they get the best quality money can buy. But in the digital world, this prediction is not fulfilled automatically.
From the perspective of pure image quality, the Leica DMR is a first-class product, but performance-wise it does not reach the level of the best Nikon and Canon DSLR's. If this report sounds a bit critical, then this is true, as Leica in their marketing efforts have claimed superior performance on the level of their filmbased product line.
The DMR/R9 combo however is too much of a hybrid construction to be able to fully compete with the more dedicated and more homogeneously designed full-digital products of the competition.
The DMR as it stands now is ripe for a Mark II version, but given the slow pace of the developments at Leica we have to wait and see. Let us hope that the development of the announced Md-camera has room for ongoing improvements. To market a product at the end of 2006 with the technological state-of -the art of 2004, may be acceptable in film-based photography with its different set of requirements, but in the world of silicon and software product cycles and generations are measured en defined in years, not decades or even five-year periods.
