Monday, November 17, 2008

DxO: Rank Your Camera?

(Updated) DxO has announced their new RAW Analyzer statistics website with great flourish today, and, of course, you can brazenly go look up Pentax K10D and K20D statistics (link). How do they fair? Well, it's still hard to tell. Little bits of control information, like what lenses they've used, aren't freely available as yet on the site. This is a meant to be a taste of's overall goal to make the DxO Analyzer the standard benchmark test for cameras and lenses for consumers. The gist is that they claim to have better testing protocols and they now evaluate RAW files before they've been rendered into jpgs. This "sensor" testing is much more advanced for measuring image quality than MTF charts rendered directly by the camera or by 3rd party software. They also claim to be bias free (unless that bias is to hype DxO's credentials). So, rather than rely on a small number of published reviewers to use their DxO Analyzer software and popularize the benchmark (like Luminous Landscape does), they're making their results public to help enlighten more of us.

I wonder, is this generous to conusmers? Or is this a marketing ploy to underscore that they have fastidious dedication to helping professionals make the most of their RAW files and sell wares like DxO optics Pro? Or is a necessary process of natural selection, throwing cameras into a pixel pit fight so that the strongest survives, helping the evolutionary process? The best scientific reporting must be done with complete transparency and unfortunately the proprietary Analyzer software isn't transparent to the general public, although you can taste it online (link) and much of the numbers and control information eventually come to light by reviewers privileged to see Analyzer results. However, what is good is that the site does offer up quick comparisons and interesting metrics in a novel way (quite different from ye olde DP Review comparisons). And the interactive illustrations that overlay on certain graphs are obviously valuable to quickly grasp the differences in ISO performance. The strongest points of the Analyzer software have always been its graphing abilities.

Then again, for all the belabored statistics gathering and graphing, it's still pixel peeping at the worst level - feeding paranoia for the insecure. The clear winners are those cameras with the best scores, which is not at all the same as a review of the best features for someone's particular needs. And it's not always evident on the site what lens combinations are being used, which is information the Analyzer software is terrific at comparing. Nor are they freely publishing information about special settings, like the K20's enhanced dynamic range ISO mode. And, of course, there will never be metrics for why most people can't shoot like Cartier-Bresson, a man who never used a Nikon D3. (aside: Will there ever be a site ranking photographers based on the dynamic range and ISO performance of shots taken by professional artists?) As an after dinner mint, please read Mike Johnston's post "Comparisons Are Odius" at The Online Photographer, where he grins happily with scientific proof that psychologist researchers have found how specifications are no substitute for visual taste.

Now, this is not to say the DxO don't make good software.  They do. All the information they carefully collect to make profiles and tweaks for their RAW conversion software is something many of us want to learn more about. And many professionals rely on such data to inform purchases of gear worth as much as a sports car. Which is why it's valuable and the good stuff comes with a price tag. To DxO's credit, it's terrific that they feel confident enough to share some of it online to try to set up a standard for reviewing. It's just that if they were truly impartial, would they have set up a Vegas-style ranking graph with the various cameras? I don't think so. They would have made more of the camera metrics freely available, sacrificing a bit of revenue in the short term for more good will. I think that any standards based methodology needs to be openly accessible, not just to journalists, and despite the generosity of DxOmark, the Analyzer isn't free.


Odyn said...

Their kung-fu very strong.

Pentax K20d has different (lower) marks than Samsung GX-20 :-))

Odyn said...

BTW... metodology explained here:

Valencia said...

I recently came accross your blog and have been reading along. I thought I would leave my first comment. I dont know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.


AC Adapter Sony AC-L200 AC-L25A AC-L25B said...

AWESOME!That's great.

All Labels For This Site (Alphabetical)

© 2006-2011 by OK1000 Pentax Blog. All rights reserved. OK1000 Pentax Blog is not affiliated with Pentax.