There is a quite strong and (on face value) convincing argument for abandoning the scientific/lab analysis of the performance of lenses as stand-alone units or in combination with film or sensor performance. The argument goes like this: when a lens is good enough for the tasks that a photographer wants to accomplish and when this lens suits the style of the photographer there is no reason to check the absolute performance of the lens. The inherent image quality of the lens is made relative to the demands and views of the photographer. The yardstick or benchmark for the image quality is defined relative to the personal requirements of the photographer.
What, you may wish to ask, is the value of someone’s opinion, even if it comes from a highly esteemed or successful photographer, about a camera or a lens? If HCB (to give some example) would note that some lens he uses, let us say the Summarit 1.5/50mm, suits his photographic style and he is very happy with this lens, what can we conclude about the true or real-life performance of this lens?
It is true that the current practice of comparative testing (using test charts and some analysis software or using an elaborate three-dimensional setup with many solid objects) has been over-hyped and has lost much of the relevance it potentially might have. It makes no sense to note that lens A has 2123 lp/ph at the edge and at aperture x and lens B at the same position and aperture records 2187 lp/ph. It also makes no sense when one notes during a pixel-peeping session that there is a slight amount of color fringing at some unsharpness edge in the background and concludes that the lens in question is now less good than it could be or as might be expected by the reviewer. In this case the yardstick is an unrealistic ideal of optical performance. What we have now is rather depressing when we are interested in the true performance of a lens: counting line-pairs does not work, personal views are too individual and subjective, pixel-peeping is a dead-end. The practical photographer may repeat the remark that one is not interested in an assessment of factual image quality as long as the lens suits the personal style and requirements of the photographer. But imagine that we have two photographers one who asserts that the lens suits him well and another one who claims that this same lens is for him disappointing.
The only solution would be to establish the true performance of the lens. We have to make one important addition to this sentence and that is hat we want to establish the true performance that is relevant for the photographic practice in general. There are several aspects to consider: the image quality relative to the range of apertures, the image quality relative to the magnification or object distances (close, medium, infinity) and the image quality relative to the film or sensor area (center, corner, edge). The image quality itself might be defined as the definition of fine detail (contrast at the edges of textural detail) and the loss of contrast due to flare and inherent image aberrations. No lens is perfect and every designer will make a reasoned balance between the many conflicting demands. Any lens review or test should explain or demonstrate these choices and the effect they have on the inherent image quality and present the facts in such a way that the user can make an informed decision based on these facts.