Comparing FF vs. APS-C

[Pieter]

I really admire your knowledge and highly inteligent testing setups, it

[Fast Glass]

Yes there is an equivalent, the 1200mm f/5.6 L. But that thing is a beast and costs as much as a small house in the county!


Here Bryan reviews it www.the-digital-picture.com/.../Canon-EF-1200mm-f-5.6-L-USM-Lens-Review.aspx.


Enjoy. Oh, and please wipe off the drool when your done....[]


John.

[HDNitehawk]

Jon, Rick, how would you suggest I set that up? I'm thinking of shooting the ColorChecker, using the same lens for equal transmittance, but...the 100 L Macro and change distance, or the 70-200/2.8 II and zoom to compensate for crop?). Manual exposure, same aperture for both cameras (since sensor size doesn't affect exposure), and a flat target obviates DoF. What else?

I think you're making it too complicated. Set your cameras so they take the same picture (as you have done with different lenses and the same aperture). Give them both the same amount of light (ie, same lighting, same shutter speed). Then you have a fair comparison. I would suggest using manual mode under the same lighting and no flash, and setting the ISO so the pictures look similarly bright or so the histograms look similar (ISO does not affect photon noise).


Transmission will be different because the lenses are different, but I don't think that is a biggie, and I don't see a way around it (using the same lens on different cameras with different f/ settings will not give exactly the same transmission percentages eithe

John


Using the same lens and adjusting distance should work, If you are just checking noise. I think low even light would be the best since you are trying to make noise.


I think Jon's suggestion would work. But how are you going to quantify the noise. And would there be some ratio that one shows more noise than the other, for instance the 7D would have 1.6x the amount of noise of the 5D? Which I am sure will not be the case.


Rick

[Jon Ruyle]

Using the same lens and adjusting distance should work, If you are just checking noise. I think low even light would be the best since you are trying to make noise.


I think Jon's suggestion would work. But how are you going to quantify the noise. And would there be some ratio that one shows more noise than the other, for instance the 7D would have 1.6x the amount of noise of the 5D? Which I am sure will not be the case.

You and John are right- using the same lens and stopping down will work fine. Plus, if you only want to measure noise, it is better to use the same lens (I take back what I said about stopping down changing the transmission percentages: I thought about it some more and I think that using the same lens at different f stops will give very close light transmission rates)


Quantifying noise is tricky. The guys at dpreview tried it and failed, and came to the wrong conclusion that cameras with smaller pixels have more noise. I suspect that this is because they did something like "take a picture of a grey card and compute the standard deviation of the pixel values". The problem with this method is that vignetting will cause increased standard deviation (it is easy to correct for this- but it must be done)

[neuroanatomist]

<span>I set up an additional test scene to look more
specifically at noise and dynamic range differences between the 5DII and
7D.<span> For this test, I used the EF
70-200mm f/2.8L IS II, using the zoom to compensate for the FOV difference
between sensor sizes.<span> Since the goal was
to look at ISO noise, I took &lsquo;different pictures&rsquo; in the sense that the DoF was
different because I used the same aperture setting for both bodies &ndash; that
served to keep exposure constant for all of the images.<span> The test setup looked like this:


<span>[img]/cfs-file.ashx/__key/CommunityServer-Components-UserFiles/00-00-00-35-15/5DIIvs7D-Setup.jpg[/img]


<span>
<span>Lighting was from a pair of 150 W-equivalent gooseneck
lamps (room lights were off).<span> Relevant
features within the image frame (shown by the magenta box) of the test setup
included:

• Background: ISO
  12233 chart (an enhanced version meeting the ISO 12233 spec but adding greater
  LW/PH resolution and features for SFR analysis)
• Lower left: X-rite
  ColorChecker Passport
• Lower middle:
  Datacolor SpyderCube
• Lower middle:
  Textured napkin (behind SpyderCube) for highlight detail
• Right side:
  Stouffer 21-step transmission wedge (each step is 0.5 stops)
• Lower right: High
  Resolution Target NIST 1010A &ndash; this is a chrome-on-glass microscope slide used
  for calibration of transmitted light image analysis systems

<span><span>
<span>The last two items were backlit by a fluorescent light
pad.<span>



<span>
<span>The cameras were set to Manual Mode.<span> Two exposure settings were used, and in both cases, exposure was right at the threshold of clipping the highlights:

• 1/60 s,
  f/5.6 for shots with the gooseneck lamps on, Live View manually
  focused on the ISO 12233 chart, WB set to tungsten
• 1/40 s, f/5.6 with the goosenecks off (so
  only the backlit step wedge and microscope slide were illuminated), Live
  View manually focused on the microscope slide, WB set to fluorescent

<span><span>
<span>Shots were taken at increasing full-stop &lsquo;native&rsquo; ISO
settings (avoiding the &lsquo;tweener&rsquo; ISOs).<span> As
ISO was increased, exposure was kept constant by the addition of B+W ND filters
(1-, 2- and 3-stops, stacked as needed for a full 6-stop range compensating for
the range of ISO 100-6400).<span> One thing I
noticed &ndash; the B+W #103 ND filter isn&rsquo;t a full three stops, more like 2.67
stops.


<span><span><span>
<span>The first set of 100% crops shows noise on the
ColorChecker:


<span><span><span><span>[img]/cfs-file.ashx/__key/CommunityServer-Components-UserFiles/00-00-00-35-15/5DIIvs7D-Noise-Composite.jpg[/img]


<span><span><span><span>
<span>Results are pretty much as expected, perhaps
even a little better &ndash; I expected the 7D to perform worse than the predicted
1.3-stop difference due to FF vs. APS-C, but it seems to be just about the
predicted difference, at least by qualitative evaluation.


<span><span><span><span><span>
<span>The second set of 100% crops shows highlight
detail, and also some sharpness differences (ignore the resolution value of 30
lw/ph since the framing of the ISO 12233 chart in this setup is not intended
for resolution testing):


<span><span><span><span><span><span>[img]/cfs-file.ashx/__key/CommunityServer-Components-UserFiles/00-00-00-35-15/5DIIvs7D-Highlight-Composite.jpg[/img]


<span><span><span><span><span><span>
<span>Highlights are well preserved throughout.<span> I won&rsquo;t post them, but the series with the
goosenecks off and only the backlit step wedge shows that the dynamic range is
similar between the two cameras &ndash; about 9 stops on the step wedge, becoming a
bit less at the upper end of the ISO range.<span>



<span><span><span><span><span><span><span><span>On e interesting &lsquo;feature&rsquo; is the diagonal RGB
color banding that shows up on the 30 lw/ph lines in the 7D images, but not the
5DII images.<span> It appears to be partly a RAW
conversion artifact &ndash; the image below is a screenshot of the RAW file and the
DPP-converted JPG opened side by side in Preview (the Mac OS X viewer app), and
the banding is not as obvious in the RAW file (left), although it's present if you look carefully (as lighter diagonal bands, without the color artifact), especially if you rapidly scroll the image up and down on the screen.


[img]/cfs-file.ashx/__key/CommunityServer-Components-UserFiles/00-00-00-35-15/5DIIvs7D-ColorBanding.jpg[/img]


<span><span><span><span><span><span><span><span> As usual, comments, feedback, and re-interpretations are welcome!


<span><span><span><span><span><span><span><span>--John

[HDNitehawk]

Nice detail work again John. I am seeing no suprises here.


+1 for the 5D II in this test


From personal use this is my perception: the dynamic range would be similar in the two bodies (7D and 5D II). The 1D IV seems to have an edge on the other two in this regard.

[Jon Ruyle]

To my eye, the 5D is just about a stop better. Comparing the 5DII image to the 7D one level up, sometimes one side looks nosier to me, sometimes the other. I

[HDNitehawk]

Of course, the test isn't exactly even: comparing 100% crops is a little unfair to the 5DII because it has more pixels. This is a difference of just less than 1/4 stop (about 0.23 stops). So if you think the tests reveal a 1 1/3 stop difference, there is really more than a 1 1/2 stop difference.

I am not sure you could do an even test. One that John did a while back we compared a 5D shot croped to match the 7D framing to show the 7D could have IQ as good as the 5D II when using your longest telephot lens. In that test the 7D in some respects matched or exceded the 5D and in a few aspects it trailed. You could say this test wasn't fair as well for the same reason, the 7D's pixel count over the croped area was higher than the 5D II's.


I think for an overall comparison you would have to try several diffrent senarios. So far all of the senarios I have seen the 7D trailed the 5D II in noise. I think every senario you would try would favor slightly one body or the other.


Here is another thought, that I do not know the answer on. Could this be a lens vs light thing instead of a sensor vs sensor issue?

[Jon Ruyle]

I am not sure you could do an even test.

I think you can come close. John's test seems pretty good to me, provided you normalize for pixel size.

<span>One that John did a while back we compared a 5D shot croped to match the 7D framing [...]You could say this test wasn't fair as well for the same reason, the 7D's pixel count over the croped area was higher than the 5D II's.

Agreed.

[Daniel Browning]

Thanks for this test, John!