Quote Originally Posted by neuroanatomist View Post
Honestly, everyone should take a step back and consider what the DxOMark scores mean - and don't mean. They are quite transparent about their scoring metrics and methods, and there's no reason to suspect there's collusion or favoritism occurring. At the same time, it's important to remember that the sensor score is just that - a score of the sensor itself, not a 'camera score'.

Agreed.


Quote Originally Posted by neuroanatomist View Post
Furthermore, keep in mind that all of their overall scores are based on reducing the image to 8 MP -

OK, after this I pretty much disagree with the point you're making here. Please indulge another one of my car analogies to try to help make it clear why I think you're driving after the wrong point. What would you think if I said the following:


Keep in mind that all of Motor Trend's road noise scores are based on driving the car at 80 MPH - thus, the greater the top speed the more you can slow down from the top speed, which means lower apparent road noise. That lower "artificially improved" road noise means apparently better accoustic performance.


Note that Motor Trend does provide the non-normalized (noise at top speed) data, they just don't use those data to calculate the overall scores, the rationale being that normalizing to 80 MPH allows appropriate comparisons. In one sense, it does - if you're going to drive exactly 80 MPH all the time, then their scores actually apply pretty well.


At 80 MPH, the Cadillac with 40 dB of road noise 'beats' the Pinto with 60 dB. In the comparison without normalizing to 80 MPH, the Cadillac's advantage pretty much entirely disappears.

Wouldn't you think I was crazy? There's just so many things wrong with that. Yes, the Cadillac *does* have higher road noise, but only when you drive it at speeds the Pinto can't even dream of. At any *overlapping* speed, the Cadillac mops the floor with it. There's nothing "artificial" or "apparent" about it. So why count the *possible* higher speed of the Cadillac as a negative? It can do everything the Pinto can do, and better. Plus it can do some stuff the Pinto can't.


Comparing sensor noise at 100% maximum spatial frequency makes about as much sense as comparing road noise at 100% maximum car velocity. (That is, none at all.)


Quote Originally Posted by neuroanatomist View Post
[...], the rationale being that normalizing to 8 MP allows appropriate comparisons. In one sense, it does - if you're going to print 8x10" images all the time, then their scores actually apply pretty well.

No. You can pick *any* overlapping pixel count and the performance differences would be the same: 22.3 MP, 8 MP, 1 MP, it doesn't matter. So you don't have to print 8x10" exactly to notice the same difference in performance -- the difference is always there.


Furthermore, there are many additional benefits of higher resolution that DxOMark does not factor into their comparison. First, if you actually give people a choice between "less noise + less resolution" or "more noise + more resolution" (where the noise power per spatial frequency is the same in both), they will pick the latter every time. Even people who *think* they hate noise. Lower resolution images just look like high-res ones with NR applied. They are never as good.

Try taking any noisy image. Downsample it to to one quarter of the spatial resolution (thus reducing noise power significantly), then resample it back up to the original pixel count. Now compare it with the original. The downsampled one has much lower noise power, but I bet you *anyone* would pick the high-noise high-detail image as their preference.

On top of that, you also have to factor in the OLPF. At all spatial frequencies above 80% of the 5D2 (i.e. more than 14 MP), the contrast is going to be reduced by the OLPF, while the D800 will not (because it's OLPF does not kick in until much higher frequencies). To compensate you will have to either increase sharpening, which increases noise, while the D800 will not require any sharpening to reach the same contrast level.

So even if the D800 had the *same* score as the 5D3 -- that is, if you got the same exact noise level after downsampling to 22.3 MP, the D800 would *still* be superior in any comparison at 14 MP or above. Below 14 MP, they would be the same.