5dIII review at DxOMark

[dsiegel5151]

I went to their compare thingy (there's a link to their compare tool at the end of the last page). I changed out the 5DmkII for a D800, clicked on measurements, and looked at all the pretty graphs comparing the D800 and the 5DmkIII.

The only difference the graphs show is that there is more dynamic range on the D800 for ISOs under 800. For everything else, they're pretty much equal. The 5DmkIII even has slightly more DR at high ISO.

I'm not sure how this ends up becoming a much higher low-light value for the D800. Low light when using ISO 100 and long exposures?

This is why I quit reading DxO: http://www.dxomark.com/index.php/Cam...(brand2)/Nikon

I don't see how these measurements give one sensor a score of a 95 and the other a score of an 81.

Here's another example: http://www.dxomark.com/index.php/Cam...brand3)/Pentax

Notice that the Pentax K5 has scored an 82 overall; however, it is inferior to the 5d Mark III in ever category except Dynamic Range at low Iso settings. Thus, am I to infer that Dynamic Range at low Iso settings is the only measurement that matters when rating a sensor?

[HDNitehawk]

I think the same way about DxO’s reviews.

What would be helpful to read is if we could talk Daniel in to finding a buddy with a 5D III, then he take his D800 and the lens that works on both cameras, and give us a side by side sample review. (big hint hint…)

[Daniel Browning]

This is why I quit reading DxO [...] I don't see how these measurements give one sensor a score of a 95 and the other a score of an 81.

It's pretty simple. You measure three areas of performance (DxO's portrait, landscape, and sports scores) and then average them together. The D800 sensor did better in all three, so it had a higher combined score.

Here's another example: [...] Notice that the Pentax K5 has scored an 82 overall; however, it is inferior to the 5d Mark III in ever category except Dynamic Range at low Iso settings. Thus, am I to infer that Dynamic Range at low Iso settings is the only measurement that matters when rating a sensor?

No. Say Johnny gets 65% in Chemistry, 65% in English, and 100% in Math. While Sally gets 70% in all three. Sally beat Johnny in two out of three subjects, but Johnny's GPA is still higher. Are you to infer that Math is the only subject that matters when calculating someone's GPA?

Any time you try to reduce something down to a single number, it will *not* reflect the full reality and complexity of the situation.

Summarizing something as complicated and nuanced as sensor performance into a single number is impossible. But a lot of people *want* a single number, because they don't have the time or inclination to learn all that stuff. So DxO provides the solution for everyone.

At one end of the spectrum (technically-minded folks), it provides the full charts and data so you can go in and see how it really does for yourself. Make up your own metrics that reflect what you do with the camera, and use their data for it.

Towards the other end of the spectrum, DxO has chosen three particular measurements, that while they may not match your shooting exactly, they do reflect the taste and perception of many photographers. You merely choose how important each of these three are to you and weight them yourself (for one, low light may be twice as important as dynamic range and color depth; for another photographer, the reverse).

Then at the very end of the spectrum you have folks for whom even three numbers is too much. They only have time or inclination to compare based on a single number. DxO doesn't know how important the three factors are for every person, so they just weight them equally. I think that's a good choice, even though it doesn't match the weighting that I would use for myself, personally.

[dsiegel5151]

It's pretty simple. You measure three areas of performance (DxO's portrait, landscape, and sports scores) and then average them together. The D800 sensor did better in all three, so it had a higher combined score.



No. Say Johnny gets 65% in Chemistry, 65% in English, and 100% in Math. While Sally gets 70% in all three. Sally beat Johnny in two out of three subjects, but Johnny's GPA is still higher. Are you to infer that Math is the only subject that matters when calculating someone's GPA?

Any time you try to reduce something down to a single number, it will *not* reflect the full reality and complexity of the situation.

Summarizing something as complicated and nuanced as sensor performance into a single number is impossible. But a lot of people *want* a single number, because they don't have the time or inclination to learn all that stuff. So DxO provides the solution for everyone.

At one end of the spectrum (technically-minded folks), it provides the full charts and data so you can go in and see how it really does for yourself. Make up your own metrics that reflect what you do with the camera, and use their data for it.

Towards the other end of the spectrum, DxO has chosen three particular measurements, that while they may not match your shooting exactly, they do reflect the taste and perception of many photographers. You merely choose how important each of these three are to you and weight them yourself (for one, low light may be twice as important as dynamic range and color depth; for another photographer, the reverse).

Then at the very end of the spectrum you have folks for whom even three numbers is too much. They only have time or inclination to compare based on a single number. DxO doesn't know how important the three factors are for every person, so they just weight them equally. I think that's a good choice, even though it doesn't match the weighting that I would use for myself, personally.

I'm just never going to get it. I noticed that the Nex7 scored the same score as the 5dIII. The Nex7 dominated the 5dIII in dynamic range at 2 iso levels (a little over one stop Ev). In pretty much every other category at every iso, the 5dIII dominated the Nex7 (http://www.dxomark.com/index.php/Cam.../(brand2)/Sony). But b/c the Nex7 had a little over one stop better dynamic range at two isos, it tied the 5dIII? It makes sense to me to combine and average percentages in terms of school grades. We all know what these percentages indicate. Is it really appropriate to combine and average three different sensor measurements into one overall score? I'm sure that DxO has some sort of conversion to make all the measurements relative (or are they just relative to some optimum?), but why is a little over 1 stop dynamic range at two iso settings weighted so heavily, especially at low iso levels? At five higher iso levels, the 5dIII dominates the Nex7 by approximately one stop of dynamic range. How is it possible that these sensors score the same?!?! It makes my head want to explode; but fortunately, I'm never buying either of these cameras, so it really doesn't matter.

[Daniel Browning]

It makes sense to me to combine and average percentages in terms of school grades. We all know what these percentages indicate.

Personally, I think combining and averaging scores tells you just as much about a student as it does about a camera (which is to say, not very much). A 4.0 GPA may seem impressive on the surface, but what if all the classes are basket-weaving and the like? Or it's from University of Nigeria (Mailorder Dept). A 2.5 may not seem as good, but what if it's for classes that are 6 years ahead of the student's level? Or what if the GPA is poor but the student is the best pianist in the world, or an Olympic gold medalist? The number is only useful if the way that it was calculated also corresponds with a way that will be useful to you. Often times they don't match up, so it's necessary to have more information than one number. If you want to know which student is the best at math, the GPA may not help.

Is it really appropriate to combine and average three different sensor measurements into one overall score?

Yes, just as much (or more) as any single number score is appropriate in any type of summarization of performance or value -- like rottentomatoes, amazon review stars, etc. Oblig. xkcd: http://xkcd.com/937/

Personally, I would never use the single DxOMark score, because a single number really cannot tell anyone enough to make a good decision. But if someone asked me to rank sensors by a single performance number, without knowing what the sensor would be used for, I would probably use DxO's. Incidentally, their three measurements (Landscape, Portrait, and Sports) correspond very closely to the three primary performance metrics that image sensor designers optimize for (read noise, full well, and quantum efficiency).

I'm sure that DxO has some sort of conversion to make all the measurements relative (or are they just relative to some optimum?),

The scores are open-ended, not relative to some optimum. If I recall correctly, they are normalized to "stops" (1 bit of color depth is 2/3 stops), for photographers.

but why is a little over 1 stop dynamic range at two iso settings weighted so heavily, especially at low iso levels?

Why isn't it weighted *more* heavily? To me personally, low iso dynamic range is even more important than the weighting that DxO gives it. In fact I switched from Canon to Nikon primarily for low ISO dynamic range.

Doesn't that illustrate the point? Everyone has different preferences and desires. You don't care so much for low ISO dynamic range, so you'd prefer it to be weighted less. I'd prefer it to be weighted more. DxO had to pick *something*, probably what they thought the average photographer would think.

I know plenty of photographers who have *never* changed the ISO setting from 100 (in fact I was at his house this evening), nor ever used more than 6 stops of dynamic range. For them, the portrait score is pretty much all they need.

At five higher iso levels, the 5dIII dominates the Nex7 by approximately one stop of dynamic range. How is it possible that these sensors score the same?!?!

Because the dynamic range at high ISO does not factor into the *landscape* score. Most landscapes are not shot at high ISO.

It makes my head want to explode; but fortunately, I'm never buying either of these cameras, so it really doesn't matter.