Comparing FF vs. APS-C

[HDNitehawk]

I chose different ISO's in an attempt to compensate for the effect of sensor size on noise (and chose incorrectly by 1/3 stop). However, to try and keep things on relatively even footing, shutter speeds were kept constant as aperture changed, and were similar on the two cameras - to compensate for the narrower apertures, I increased Speedlite power. Here, too, I goofed...I simply used the C.Fn to set the sync speed to the max, meaning the 5DII shutter speed was 1/200 s throughout and the 7D was 1/250 s throughout - again, a 1/3-stop difference, but unfortunately in the same direction as my ISO choice.


So, in a way, I inadvertently stacked the noise deck in favor of the 5DII by 2/3 of a stop. Still, to me, there is more than 2/3 of a stop worth of extra noise in the 7D images. I'm starting to believe in Rick'sconspiracy theory... Looking at Bryan's noise tests in the 60D review, the 60D appears to have slightly less ISO noise(a subtle difference, but visible),even at lower ISOs, compared to the 7D. This may represent improved technology (despite the fact that the same sensor is used), or may be consistent with Rick'sconspiracy theory - they didn't have to dumb down 60D as much as the 7D in terms of ISO performance, because they dumbed it down in other ways (e.g. no AF microadjustment

John


I am not sure what a fair test would be considering the variables. At some point you are going to have to adjust either ISO or Shutter Speed or your Flash. Perhaps a comparison where you have no flash and equal lighting, and choose the shutter and ISO that you would choose for each camera in that situation. That would give you a true read of how that camera would perform with your own personal shooting style. For instance with the set up you used I might choose a moderate ISO and longer shutter speed because you are on a tripod shooting a static object. In the back of my mind though I suppose it would still be biased, because I wouldn't let the 7D ISO ever get as high as the 5D II. Still this wouldn't truly be a fair comparison, because that is what the 5D II is meant to do, where the 7D is more action oriented.


I think regardless of the test, you are going to get similar results.





Rick

[Jon Ruyle]

am not sure what a fair test would be considering the variables. At some point you are going to have to adjust either ISO or Shutter Speed or your Flash. Perhaps a comparison where you have no flash and equal lighting, and choose the shutter and ISO that you would choose for each camera in that situation.

The test is fair only if both cameras get the same amount of light. No flash and equal lighting as you suggest would be good, but the shutter speeds *must* be the same.

[HDNitehawk]

The test is fair only if both cameras get the same amount of light. No flash and equal lighting as you suggest would be good, but the shutter speeds *must* be the same.

Wouldn't you have to disregard the DOF difference of the two lenses. Pardon my ignorance here I am making this assumption, but to get the proper identical exposure even the aperture would be have to be the same in your scenario. Your subject would look different because of the 1.6x deeper depth of field so while the comparison would have the same framing the general feel of the picture would be different.

[neuroanatomist]

Jon, the increased Speedlite power was not to compensate for the sensor differences, but to compensate for the 2.5- and 5-stop aperture decrements. In theory, a decrease in aperture and a corresponding increase in available light, holding shutter speed constant, means the same amount of light hitting the sensor, right? Having said that, I did rely on the E-TTL II flash metering, and there were some exposure differences (within 1/2 stop), and color temp differences due to the balance of flash vs. ambient (tungsten/halogen), which I equalized in the RAW conversion using the ColorChecker as a reference.


Rick, my experience matches yours as far as the 5DII and 7D go - the 7D is just plain noisier, significantly so.


Honestly, I wasn

[Jon Ruyle]

n theory, a decrease in aperture and a corresponding increase in available light, holding shutter speed constant, means the same amount of light hitting the sensor, right?

When you say "decrease in aperture" I think you mean "increase in f number". In your tests, the apertures were the same, as was appropriate (f numbers were different because the focal lengths are different). If the apertures (not f numbers) are the same, effective focal lengths are the same, lighting is the same, and the shutter speeds are the same, each sensor is getting the same amount of light.

I now just take for granted that the 7D is noisy by comparison.

That depends on how you do the comparison. At the same ISO, the of course the 7D has far more noise. But if you take the same picture with both cameras, I think the 7D will have less noise.

Honestly, I wasn't thinking much of noise when I set this up

I wasn't either when I first looked at it... it wasn't until you said the 7D was noisier that I thought... "wait, something is wrong...." (Not wrong necessarily, but not geared to compare noise.)

Jon, Rick, how would you suggest I set that up? I'm thinking of shooting the ColorChecker, using the same lens for equal transmittance, but...the 100 L Macro and change distance, or the 70-200/2.8 II and zoom to compensate for crop?). Manual exposure, same aperture for both cameras (since sensor size doesn't affect exposure), and a flat target obviates DoF. What else?

I think you're making it too complicated. Set your cameras so they take the same picture (as you have done with different lenses and the same aperture). Give them both the same amount of light (ie, same lighting, same shutter speed). Then you have a fair comparison. I would suggest using manual mode under the same lighting and no flash, and setting the ISO so the pictures look similarly bright or so the histograms look similar (ISO does not affect photon noise).


Transmission will be different because the lenses are different, but I don't think that is a biggie, and I don't see a way around it (using the same lens on different cameras with different f/ settings will not give exactly the same transmission percentages either)

[Pieter]

I really admire your knowledge and highly inteligent testing setups, it

[Fast Glass]

Yes there is an equivalent, the 1200mm f/5.6 L. But that thing is a beast and costs as much as a small house in the county!


Here Bryan reviews it www.the-digital-picture.com/.../Canon-EF-1200mm-f-5.6-L-USM-Lens-Review.aspx.


Enjoy. Oh, and please wipe off the drool when your done....[]


John.

[HDNitehawk]

Jon, Rick, how would you suggest I set that up? I'm thinking of shooting the ColorChecker, using the same lens for equal transmittance, but...the 100 L Macro and change distance, or the 70-200/2.8 II and zoom to compensate for crop?). Manual exposure, same aperture for both cameras (since sensor size doesn't affect exposure), and a flat target obviates DoF. What else?

I think you're making it too complicated. Set your cameras so they take the same picture (as you have done with different lenses and the same aperture). Give them both the same amount of light (ie, same lighting, same shutter speed). Then you have a fair comparison. I would suggest using manual mode under the same lighting and no flash, and setting the ISO so the pictures look similarly bright or so the histograms look similar (ISO does not affect photon noise).


Transmission will be different because the lenses are different, but I don't think that is a biggie, and I don't see a way around it (using the same lens on different cameras with different f/ settings will not give exactly the same transmission percentages eithe

John


Using the same lens and adjusting distance should work, If you are just checking noise. I think low even light would be the best since you are trying to make noise.


I think Jon's suggestion would work. But how are you going to quantify the noise. And would there be some ratio that one shows more noise than the other, for instance the 7D would have 1.6x the amount of noise of the 5D? Which I am sure will not be the case.


Rick

[Jon Ruyle]

Using the same lens and adjusting distance should work, If you are just checking noise. I think low even light would be the best since you are trying to make noise.


I think Jon's suggestion would work. But how are you going to quantify the noise. And would there be some ratio that one shows more noise than the other, for instance the 7D would have 1.6x the amount of noise of the 5D? Which I am sure will not be the case.

You and John are right- using the same lens and stopping down will work fine. Plus, if you only want to measure noise, it is better to use the same lens (I take back what I said about stopping down changing the transmission percentages: I thought about it some more and I think that using the same lens at different f stops will give very close light transmission rates)


Quantifying noise is tricky. The guys at dpreview tried it and failed, and came to the wrong conclusion that cameras with smaller pixels have more noise. I suspect that this is because they did something like "take a picture of a grey card and compute the standard deviation of the pixel values". The problem with this method is that vignetting will cause increased standard deviation (it is easy to correct for this- but it must be done)

[Jon Ruyle]

but to get the proper identical exposure even the aperture would be have to be the same in your scenario.

Using the same aperture would defeat the point, because the one with the larger sensor would have a narrower DOF... that's a *different picture*, thus not comparing apples to apples.


What John is trying to do (and what seems to me the only reasonable thing to do) is to compare two different camera/lens combinations taking the *same picture*. So they should have the same aperture and same focal length (thus same DOF and same perspective and angle of view). Obviously the f numbers will be different.


If we want to compare noise, we should use the same shutter speed and similar lighting. If we use a slower shutter speed with one camera, we're giving that camera more light and thus giving it an advantage. If we use a flash and allow the different meters to decide how bright to make the flash, the cameras are getting different amounts of light.


There is no need to think about ISO at all, since it has no effect on photon noise (it can affect read noise, but I think this is outside the scope of the ff vs crop comparison). The only consideration for ISO is that it should be set so that both pictures are exposed reasonably. If we do this, the camera with the larger sensor will have a higher ISO, but will have no inherent advantage or disadvantage because of sensor size.


If you are comparing your 1DIV or 5DII vs the 7D using the same iso, of course the 5DII and 1DIV with their larger sensors will do better.