How Do You Test a New Lens?

[HDNitehawk]

I have never tested in this manner. I try to eliminate the variables so I can focus on one thing. I would check the lenses side by side in manual focus in live view and compare the results. That will tell you how your copy compares to your 24-105 for sharpness.

http://www.the-digital-picture.com/R...mp=4&APIComp=0

At the same aperture of f/4 the 24-70 should be better.

From the sharpness test -15 isn't the setting. I liked the previous results which peak at f/2.8. I think those settings were closer to what it is really trying to do.

70mm should be the easiest focal length to test. If you are having this much trouble it leads me to believe that it should go back.
70mm is a weak point for this lens, but there are some copies that perform very well. As long as Adorama keeps exchanging I would keep trying.

The AF aperture sharpness tests I do I set up at 50x with good lighting. Then I went from +5 one step at a time to -5 and then compared results.
Keep in mind that when you do the test the camera AF once at the start of the run. Since not all AF shots are the same, one may be a bit behind and the next in front the test can be time consuming. You end up with some results that just don't fit because the AF system missed. But if you run the scenario several times you will see a pattern develop.

[neuroanatomist]

I also just ran the FoCal's Aperture Sharpness test @ 70 mm with the AFMA = -15
Attachment 2052

That doesn't look good, to me. You should be seeing a fairly flat curve. Here's the aggregated data from FoCal:

[Kayaker72]

Thanks Rick and John. And I agree. I've decided to hold onto the lens until this weekend when I can do a few more tests (I want to try 50x distance, which is 3.5 meters vs the 2.3 m I ran my tests at, which is ~33x). But I am debating between returning it to Adorama vs sending it into Canon for repair under warranty. I have to run catch a flight, but I plan on calling canon tomorrow to discuss. I have a pretty strong lean toward returning the lens. BTW, any ideas what may be causing the dip from f/8-f/4?

[HDNitehawk]

If the lens is on the low end of IQ out of the 24-70mm II group, I think Canon would just send it back to you and say it is within their parameters.
We are a bit more critical than they are.

The reason for the fall in IQ from f/8 to f/4 is a failed test IMO. Like I said previously the camera will AF once at the start of the run, vibration and such can knock it out of focus over the course of all the shots. I had this happen several times with a similar chart, when repeating the test I would find it was just a failed test since I got other runs that were ok.


If I was not using the best AFMA point I would get results similar to the charts you posted previously.

John's chart is how it should look. Mine peaks at f/3.2 at 70mm as well.
At 24 and 35mm mine peaks at f/2.8.

[Kayaker72]

So, I am finishing up evaluating this third copy of the 24-70 II I've received. Copies 1 & 2 went back for because of an audible clicking while operating the zoom. This third copy has checked out except for some odd behavior at 70 mm (24 mm and 50 mm were fine). As I am not the most versed lens tester out there, please let me know what you think of the following.

At 70 mm, the lens requires a different AFMA to optimize sharpness for different apertures. Using FoCal at 50x distance, etc, I had the following results:

• f/2.8 = 0
• f/4 = -2
• f/5.6 = -6
• f/8 = -3
• f/11 = +3

Rather than a well defined peak, the gradient is pretty flat form -5 to +5, usually with an odd spike some place.

In another test, I line up a test with subjects at different distances and as I vary the aperture and the center of the focal plane also seems to vary (not the most elaborate set up, but multiple shots seem to confirm this).

Sharpness with this lens doesn't seem to be an issue. When I manually focus the lens it is really impressing me. Even most AF shots are very impressive.

I've already contacted Adorama and I think this is going back tomorrow. But I would be very interested if anyone knew how to fix this becuase in terms of pure sharpness, based on manual focused shots, I would love to keep this lens.

Thanks,
Brant

[HDNitehawk]

The copy I kept I was not able to get it to perform correctly with my 1D IV. I found that odd because every other lens works great with it.
With the 5D II it worked flawlessly.

This lens is so sharp I would be concerned how it performed at f/4 and below rather than above. Also how does it compare to other distances (24mm, 35mm and 50mm). If all those are consistent or close to what you get at f/2.8 at 70mm that would sway my decision to send it back or not.

If you kept the report where the F/5.6 gave you a -6 I would be curious to see the curve.

The results sure sound odd.

[Kayaker72]

Rick,

Here is the chart from the f/5.6 @ 70 mm:



For good measure here is f/2.8:


f/4:



and f/8:


BTW, the critical zone, I was taking shots every 3 AFMA units and sometimes 3 shots. Just trying to generate more data.

One more thing, most of my other lenses are AFMA +6 to +9. So I think my 5DIII is skewed into that range which makes the negative AFMA with the 24-70 II @ 70 mm that much more troublesome.

Let me know what you think.
Brant

[HDNitehawk]

Brant,

a few observations;

1, It is showing a quality of focus at f/2.8 and f/8 between 1800 and 1900. I do not know how this compares to the other focal lengths or your other lenses with your set up, but it looks like you are getting very good IQ.

2, The earlier aperture quality chart you posted supports what we see in the charts you just posted. At f/4 and f/5.6 your IQ drops. At first glance the gradual round shape of the charts makes me think they were not accurate takes, but I do not think it is the case. For some reason, as you thought before, the IQ does indeed drop at those apertures.

If I kept the lens I would use the f/2.8 setting. Possibly bias it to -1 or -2.

If I thought the IQ at f/2.8 and f/8 are sweet spots and very high and the mid apertures were still exceptional I might think about keeping it. If I thought the IQ was good at f/2.8 and f/8 and weak and soft at the other two settings I would ship it back. Still your results are not the norm and I would be debating it in my mind.

[Kayaker72]

Thanks Rick. You have pretty much summed up where I am at. Overall, the lens appears to be optically sharp. In comparing manual focused shots at 70 mm, I had printed out the ISO 12233 chart from Cornell University. I compared the 24-70 II, 24-105 and the 70-200 II. Trying to "fill the frame" with the chart, manually focused, the 24-70 was the best. I could see detail is the grain of the wood in the paper the chart was printed on. But go back to ~12ft, switched to AF, and the 24-70 II was slightly worse than the other two. Adjusting AFMA didn't really switch that, just the individual sharpness of each aperture.

I may be over thinking this (typical ). But my concern isn't necessarily where it is at now, as it is sharp. But I am trying to wrap my head around what could be wrong and how it could play out in the future. In other words, could it get worse? But, what is wrong? A loose/flawed/decentered element in the focusing group? I am not sure. Which is why I am leaning toward sending it back.

[Kayaker72]

Just an update for those interested....Copy 4 of this lens arrived on Monday....I have been putting it through a series of tests. Sharp lens. Much sharper than 24-105 f/4, including at 70 mm (unlike copy 3). Maybe slightly sharper than the 70-200 f2.8 II at 70 mm (very similar). No odd issue at 70 mm as was observed with copy 3. AF accuracy is better than previous copies, 97% at 24 mm and 98% at 70 mm.

But, there is a small bubble in the front element. My small, about normal dust sized.

I am off to work still thinking about if I will keep, return to adorama, or send to canon under warranty. Leaning toward keeping this one. I doubt a very small bubble will influence many shots. Anyone have experience with this?