Comparing FF vs. APS-C

[Jon Ruyle]

to compensate for the narrower apertures, I increased Speedlite power.

You mean you gave the 5DII more light? Or do you mean you let the meter decide?


To make a fair comparison, each camera should get the same amount of light. So the shutter speed should be the same in ambient light, or the speedlight should be set to some fixed *equal* output value rather than relying on the meter. The 5DII will probably require a higher ISO to expose reasonably.


I don't think you stacked things in favor of the 5DII by using different shutter speeds, but if you let the meter decide, you ultimately allowed the meter of each camera to decide how much light to give the sensor. This could lead to a largeish difference one way or the other (I have no idea which way the bias would be, though)


If you told the flash to give the 5DII more light to compensate for the smaller aperture, you stacked the test heavily in favor of the 5DII. (I don't think you did this, though, because the 5DII would appear brighter if you had... after all, you already mostly compensated for the narrower aperture with ISO)


Unless the DXO Mark guys screwed up, the 7D should come out ahead in this comparison. It has lower read noise and lower photon noise.

[HDNitehawk]

Unless the DXO Mark guys screwed up, the 7D should come out ahead in this comparison. It has lower read noise and lower photon noise.

Jon


I haven't looked at the DXO guys data. But just from using the cameras I have it is very evident to me that in equal situations the 7D has more noise than the 5D II. As well the 1D IV has less than the 7D. The 1D IV seems to be close to the 5D II in this respect, but I can sense a slight edge with the 5D but not enough to notice or matter.


It might be worth looking at how DXO did their comparison to see if it is legitimate.

[HDNitehawk]

I chose different ISO's in an attempt to compensate for the effect of sensor size on noise (and chose incorrectly by 1/3 stop). However, to try and keep things on relatively even footing, shutter speeds were kept constant as aperture changed, and were similar on the two cameras - to compensate for the narrower apertures, I increased Speedlite power. Here, too, I goofed...I simply used the C.Fn to set the sync speed to the max, meaning the 5DII shutter speed was 1/200 s throughout and the 7D was 1/250 s throughout - again, a 1/3-stop difference, but unfortunately in the same direction as my ISO choice.


So, in a way, I inadvertently stacked the noise deck in favor of the 5DII by 2/3 of a stop. Still, to me, there is more than 2/3 of a stop worth of extra noise in the 7D images. I'm starting to believe in Rick'sconspiracy theory... Looking at Bryan's noise tests in the 60D review, the 60D appears to have slightly less ISO noise(a subtle difference, but visible),even at lower ISOs, compared to the 7D. This may represent improved technology (despite the fact that the same sensor is used), or may be consistent with Rick'sconspiracy theory - they didn't have to dumb down 60D as much as the 7D in terms of ISO performance, because they dumbed it down in other ways (e.g. no AF microadjustment

John


I am not sure what a fair test would be considering the variables. At some point you are going to have to adjust either ISO or Shutter Speed or your Flash. Perhaps a comparison where you have no flash and equal lighting, and choose the shutter and ISO that you would choose for each camera in that situation. That would give you a true read of how that camera would perform with your own personal shooting style. For instance with the set up you used I might choose a moderate ISO and longer shutter speed because you are on a tripod shooting a static object. In the back of my mind though I suppose it would still be biased, because I wouldn't let the 7D ISO ever get as high as the 5D II. Still this wouldn't truly be a fair comparison, because that is what the 5D II is meant to do, where the 7D is more action oriented.


I think regardless of the test, you are going to get similar results.





Rick

[Jon Ruyle]

But just from using the cameras I have it is very evident to me that in equal situations the 7D has more noise than the 5D II.

By "equal situations", do you mean equal f-numbers, or equal in the sense John did his experiment (with the larger sensor stopped down)? At the same f-number, the larger sensor of course has a major advantage.

It might be worth looking at how DXO did their comparison to see if it is legitimate.

The DXO description of their tests looks good to me, but that does not mean they did not make mistakes.

[Jon Ruyle]

am not sure what a fair test would be considering the variables. At some point you are going to have to adjust either ISO or Shutter Speed or your Flash. Perhaps a comparison where you have no flash and equal lighting, and choose the shutter and ISO that you would choose for each camera in that situation.

The test is fair only if both cameras get the same amount of light. No flash and equal lighting as you suggest would be good, but the shutter speeds *must* be the same.

[HDNitehawk]

The test is fair only if both cameras get the same amount of light. No flash and equal lighting as you suggest would be good, but the shutter speeds *must* be the same.

Wouldn't you have to disregard the DOF difference of the two lenses. Pardon my ignorance here I am making this assumption, but to get the proper identical exposure even the aperture would be have to be the same in your scenario. Your subject would look different because of the 1.6x deeper depth of field so while the comparison would have the same framing the general feel of the picture would be different.

[neuroanatomist]

Jon, the increased Speedlite power was not to compensate for the sensor differences, but to compensate for the 2.5- and 5-stop aperture decrements. In theory, a decrease in aperture and a corresponding increase in available light, holding shutter speed constant, means the same amount of light hitting the sensor, right? Having said that, I did rely on the E-TTL II flash metering, and there were some exposure differences (within 1/2 stop), and color temp differences due to the balance of flash vs. ambient (tungsten/halogen), which I equalized in the RAW conversion using the ColorChecker as a reference.


Rick, my experience matches yours as far as the 5DII and 7D go - the 7D is just plain noisier, significantly so.


Honestly, I wasn

[Jon Ruyle]

but to get the proper identical exposure even the aperture would be have to be the same in your scenario.

Using the same aperture would defeat the point, because the one with the larger sensor would have a narrower DOF... that's a *different picture*, thus not comparing apples to apples.


What John is trying to do (and what seems to me the only reasonable thing to do) is to compare two different camera/lens combinations taking the *same picture*. So they should have the same aperture and same focal length (thus same DOF and same perspective and angle of view). Obviously the f numbers will be different.


If we want to compare noise, we should use the same shutter speed and similar lighting. If we use a slower shutter speed with one camera, we're giving that camera more light and thus giving it an advantage. If we use a flash and allow the different meters to decide how bright to make the flash, the cameras are getting different amounts of light.


There is no need to think about ISO at all, since it has no effect on photon noise (it can affect read noise, but I think this is outside the scope of the ff vs crop comparison). The only consideration for ISO is that it should be set so that both pictures are exposed reasonably. If we do this, the camera with the larger sensor will have a higher ISO, but will have no inherent advantage or disadvantage because of sensor size.


If you are comparing your 1DIV or 5DII vs the 7D using the same iso, of course the 5DII and 1DIV with their larger sensors will do better.

[neuroanatomist]

I agree that FF will do better than APS-C on noise. But the theory seems to suggest a 1.3-stop difference, and my impression is that the 5DII-7D differential is greater than that, which is what I

[Jon Ruyle]

n theory, a decrease in aperture and a corresponding increase in available light, holding shutter speed constant, means the same amount of light hitting the sensor, right?

When you say "decrease in aperture" I think you mean "increase in f number". In your tests, the apertures were the same, as was appropriate (f numbers were different because the focal lengths are different). If the apertures (not f numbers) are the same, effective focal lengths are the same, lighting is the same, and the shutter speeds are the same, each sensor is getting the same amount of light.

I now just take for granted that the 7D is noisy by comparison.

That depends on how you do the comparison. At the same ISO, the of course the 7D has far more noise. But if you take the same picture with both cameras, I think the 7D will have less noise.

Honestly, I wasn't thinking much of noise when I set this up

I wasn't either when I first looked at it... it wasn't until you said the 7D was noisier that I thought... "wait, something is wrong...." (Not wrong necessarily, but not geared to compare noise.)

Jon, Rick, how would you suggest I set that up? I'm thinking of shooting the ColorChecker, using the same lens for equal transmittance, but...the 100 L Macro and change distance, or the 70-200/2.8 II and zoom to compensate for crop?). Manual exposure, same aperture for both cameras (since sensor size doesn't affect exposure), and a flat target obviates DoF. What else?

I think you're making it too complicated. Set your cameras so they take the same picture (as you have done with different lenses and the same aperture). Give them both the same amount of light (ie, same lighting, same shutter speed). Then you have a fair comparison. I would suggest using manual mode under the same lighting and no flash, and setting the ISO so the pictures look similarly bright or so the histograms look similar (ISO does not affect photon noise).


Transmission will be different because the lenses are different, but I don't think that is a biggie, and I don't see a way around it (using the same lens on different cameras with different f/ settings will not give exactly the same transmission percentages either)