Those "in-between" ISOs...

[Daniel Browning]

So the capture is defined (partly) by the ISO setting - it is"hard-wired" into the result so to speak, right?

Yes. Some of the ISO settings use analog gain (amplifiers), but many of them are just digital manipulations.

I can't help thinking there must be some reason they went the way they did.

I should point out that they used to have this feature on the Canon 10D. It disappeared from all later cameras.


I can think of several possible reasons:


* Because they can. Customers accept all sorts of compromises at high ISO, even ones that are completely unnecessary (like this). Many photographers never even use high ISO. Canon knows that few people use it and the ones who do just have to be willing to accept additional unnecessary compromises.

* Because of Customer Service. Some customers will have a rude awakening when they realize that Canon has been deleting 1, 2, or more stops of their highlight headroom for no reason. Even if Canon buries the metadata feature in some custom function, and provides copious documentation, many customers will get thoroughly confused about what it is or does. Misconceptions will travel rapidly on the web about it, no doubt. Even some raw processing programs might get confused by the metadata. Adobe, for example, doesn't implement HTP correctly, it just does a linear push without preserving highlights. Other converters might not apply the push at all, so customers will wonder why it's too dark.

* Because of Management. Canon had ISO metadata in the 10D, but removed it in all later cameras. It may have been removed on purpose, by edict from Management. Software engineers wouldn't remove a feature that was doing the obviously correct thing and replace it with the obviously incorrect thing for no good reason. A good reason would be if the manager said "cripple the camera or I break-a-you-face."

* Because of Marketing. Perhaps the feature was removed because Canon wanted to save that trick for a rainy day, so they could come out with a new camera with "2 stops more dynamic range at ISO 6400!" without any development cost. If they were nefarious enough, they could sell a firmware update to all cameras going back to the 20D offering ISO 25600 as metadata, with 4 stops more dynamic range across all cameras. Sony sells $1,000 firmware upgrades on some of its high end video cameras, so there's precedent.

* Because of Engineering. It's possible that the software engineers responsible for the firmware are unaware of the issue, but given the high level of competence they've demonstrated I think that's highly unlikely. I don't think Hanlon's razor applies.

* Because of Bureaucracy. This is the most likely reason, IMHO. Many suboptimal things happen in big corporations. Maybe the 10D metadata ISO feature did not have the correct "addition of feature form" filled out, and the engineer who added it was fired because he didn't follow the correct procedure for requesting authority to add a new feature. Maybe the paperwork for removing the feature was easier than the paperwork required for leaving it in.


Who knows.

[canoli]

hmm, a number of scenarios, all sounding plausible - and thank your for your reply by the way - but they don't seem to account for what I assume is true: that it's not only easier for Canon to implement metadata ISOs, it's more profitable as well.


In your first scenario, they chose the route they did "because they can." But isn't this costing them money and angering their more knowledgeable user base? Or have I made an incorrect assumption? I'm working from the premise that "hardware ISO" (for lack of a better term) is 1) complicated to configure, requiring different strategies for different bodies, 2) demands continual R&D money, and 3) is susceptible to malfunction. Maybe #3 is no longer a concern, but it is one more electrical system that could conceivably break down.


If it'smore profitable to use metadata ISOs, then I wonder
if the reason isn't the one you listed later, the one requiring a "nefarious"
motive.. to roll out a brand new camera body featuring "FOUR EXTRA STOPS of
Dynamic Range!" That would be irresistible when the financial outlay is zero (or close to it). On the other hand, perhaps the cost, when you consider the deals and arrangements that Canon must have with RAW converter manufacturers, not to mention their own DPP...they may have to recode their products...or would they?


Getting back to the real-world implications
- if I may. Those "extra stops," the ones we lose when we use higher ISOs -
they're gone forever, right? No RAW converter can get them back because,
to the converter, they don't exist. Those HLs have been pushed off the scale and were simply not captured
during exposure. The trade off is that we get (the same amount?) footroom - we get the low range frequencies. Do I understand that all correctly?



Thanks again D.

[Daniel Browning]

Or have I made an incorrect assumption? I'm working from the premise that "hardware ISO" (for lack of a better term) is 1) complicated to configure, requiring different strategies for different bodies, 2) demands continual R&D money, and 3) is susceptible to malfunction. Maybe #3 is no longer a concern, but it is one more electrical system that could conceivably break down.

I think we have a misunderstanding. I have no problem with the hardware ISO (analog gain): ISO 200, 400, 800, and 1600. They're great and I want to keep them as an option. The only problem I have is with the software ISO (digital gain applied in camera): including the high ISO (3200, 6400), low ISO (50), and tweeners (50, 160, 250, 320, 500, etc.).


Instead of doing the digital manipulations in the camera like they are now, Canon should allow the user to do them in post.

On the other hand, perhaps the cost, when you consider the deals and arrangements that Canon must have with RAW converter manufacturers, not to mention their own DPP...they may have to recode their products...or would they?

It's so simple that it takes less than a dozen lines of Matlab code to load the entire raw file and apply the EC. In any case, Canon has no qualms about breaking compatibility and other raw converters. They even added "metadata ISO" in the form of HTP, which some raw converters ignore and others support incompletely (Adobe).

Getting back to the real-world implications
- if I may. Those "extra stops," the ones we lose when we use higher ISOs -
they're gone forever, right?

Right.

No RAW converter can get them back because,
to the converter, they don't exist. Those HLs have been pushed off the scale and were simply not captured
during exposure.

Right. Of course if it is only one or two channels that were lost, some converters will try to "guess" what the correct values would have been, but that's a separate thing.

The trade off is that we get (the same amount?) footroom - we get the low range frequencies. Do I understand that all correctly?

There is no trade off between metadata ISO and in-camera digtal ISO.
It's all negatives. You get clipped highlights, bigger files, and
reduced precision.


There is a trade-off between low analog ISO and high analog ISO: high ISO reduces headroom but increases footroom.

[canoli]

doh. Actually I didn't misunderstand, I just forgot the original subject of my thread (!) - we were justtalking about the tweener ISOs weren't we?...thanks for clarifying.


couple things:


Would you avoid using HTP if you're going to convert with ACR? I don't use HTP because I don't trust it ("understand it" is probably more accurate). The 40D manual says it "improves the HL detail," and the "dynamic range is expanded from the standard 18% gray to the brightest highlights." So that means if you meter a white dress it will actually record as white; you don't need to apply EC for white (non-specular) highlights? Anyway, since I may eventually want to start using it (bridal shots), will you share your thoughts on using HTP and then converting in ACR? Is DxO a better all-around converter? (the only other one I know besides DPP, which I don't like at all)


Something else: Does the "separate analog amplifier" used in the 5D (and the other bodies you mentioned) mitigate, or eliminate the negative effects of the 'tweener ISOs? I don't think you commented on that. (I do know there's a reason why that 5D churns out such buttery smooth, satisfyingly sharp pix - maybe it's the pixel pitch, those big ol' 8.2µm receptor sites...?)


and finally, from earlier in the thread -

I should mention that tweeners can sometimes help broken raw
converters. Canon's DPP, for example, doesn't use the correct white
point for many of their own cameras. They set it too low, ignoring
plenty of good data in the highlights completely.

Does this explain why everything in DPP looks so washed out compared to ACR? The contrast, the colors, really everything looks so much worse in DPP. This is why I said I don't like it at all. I've opened both apps side by side and have been astounded how differently they present the same image. I can tweak away and get them to match, but the ACR image begins from a superior state to my eye.


As always, thank you Daniel - as much as you're willing to write...!

[Daniel Browning]

Would you avoid using HTP if you're going to convert with ACR?

It's not ideal, but it's usable. Ideally we'd be able to reverse the +1 EC that Adobe applies behind the scenes, but I don't know how to do that. (The Exposure Compensation slider works, but it also actives the clipped highlights guessing algorithm, which is not always desirable.) It would be better to use a +1 EC with the brightness slider instead of the +1 linear (simple) EC that Adobe does behind the scenes.

The 40D manual says it "improves the HL detail," and the "dynamic range is expanded from the standard 18% gray to the brightest highlights." So that means if you meter a white dress it will actually record as white; you don't need to apply EC for white (non-specular) highlights?

Not quite. If you normally have ~3 stops between the tone that you want rendered 18% gray and "clipping", then HTP will increase it to four stops. So if the bride's skin is where you want 18% to be, and 3 stops causes a lot of the dress to blow out, then you can enable HTP and it will give you 4 stops: enough to retain that detail in the dress.

Is DxO a better all-around converter?

I think it has higher image quality (especially underexposed high ISO shots) and more control over the image, but I don't use it because Lightroom is so much faster and easier. I like Bibble a lot, I had that before Lightroom. I hear good things about Silky Pix. You might also consider some of the free raw converters. I like Raw Therapee and RPP.

Something else: Does the "separate analog amplifier" used in the 5D (and the other bodies you mentioned) mitigate, or eliminate the negative effects of the 'tweener ISOs?

Half of them do. The other half are inferior to digital gain in post because they add their own small amount of read noise to the signal. The ideal solution would be 1/3 stop gain from the primary gain amplifier, but that's not really important to me since tweeners have performance that is close enough when implemented in post with digital gain.

(I do know there's a reason why that 5D churns out such buttery smooth, satisfyingly sharp pix

The large sensor size allows lenses to be used as they were intended, which results in much higher MTF (contrast) than using the same lens on a crop camera. It also allows much thinner DOF.

- maybe it's the pixel pitch, those big ol' 8.2µm receptor sites...?)

Nope. Myth busted: smaller pixels have more noise, less dynamic range, worse diffraction, etc.

Does this explain why everything in DPP looks so washed out compared to ACR?

No, the white point bugs in DPP manifest as very slight differences in the amount of clipped data. I don't think anyone would notice it casually. The washed-out look just comes from a different default style of raw conversion. As you said, both can be tweaked to get different results, but the defaults differ by a bit.

[canoli]

Thanks Daniel, for your suggestions on a RAW converter. It sounds like they're all going to have certain strengths and weaknesses; one will perform better for a particular image and not so well on another (user knowledge being equal).


As far as my DPP vs ACR issue, if the explanation is simply "different converters use different algorithms, have different defaults," then apparently I'm in for some surprises when I check out the rest of them...

[Chuck Lee]

canoli,


You can check out SilkyPix free for 30 days. http://www.shortcutinc.com/cms/index.php


It is a fantastic converter. I use 99% of the time. It does an excellent job recovering the +1-3 ev worth of data. (even when using tweeners! LOL)


I will humbly admit though that75% of the photos from my last shootwere processed with ACR 4.6 in CS3. I used SilkyPix for the initial viewing, culling, etc. I processed the keepers but could not get my 5D outdoor flash shots to look right in print. So, I opened Bridge and went to work. The default "auto" results from ACR resulted in much better print output. Go figure. Usually for me, ACR butchers flash shots. The 40D + 50mm 1.4 @ f2.0 candid/journalist shots were fine right from SP.


It's also interesting to see ACR use brightness rather than exposure compensation to push up a dark image. I had three shots that were overexposed and it did do a -.9ev compensation automatically to bring those in line. I have to say, I was somewhat amazed at it's performance with this set of images. I usually do not like the output ACR gives.

[Chuck Lee]

Ruevski's chart correctly shows that the read noise in ADU is lower, but what really matters is SNR, and that is not improved over ISO 100 (in fact it's 1/3 stop worse).

And I'd rather have cleaner images than increased DR.

In this case, ISO 100 provides both.

Not at the same shutter speed and aperture as ISO 160. You would have to shoot ISO 100 at -.7ev to get the same speed/aperture combination. I'm willing to sacrifice a whole 1/3 stop or DR for that. The real world result is that I canhandhold at1/125th rather than 1/80th shutter speedand geta cleaner and possibly sharperimage.

[canoli]

Not at the same shutter speed and aperture as ISO 160.

Hmm, so - without getting in between you and Daniel's discussion (as it's obvious you both can talk at a higher level about this than me), I think you guys did agree that some 'tweens are better than others. 1250 over 1000, 500 over 320...(okay, that 2nd one I just made up). But there is consensus on using 1250 instead of 1000 right? And somebody mentioned that 160 on the 40D was "as good as it gets [with 'tween ISOs]."


Which 'tweener ISOs are less bad and which ones should be avoided? I realize it depends upon the camera. Personally I am particularly curious about the 40D as it's my main body for now.


One more question - is any of this relevant to amateur photography? We all want to maximize our body/lens' potential, get the best images we can, but are the differences so minute that it takes 100% views on 24" displays to notice them? Or will a photo shot at one of the especially bad 'tweeners, printed 8x10, look obviously worse than the same photo shot at a native ISO?


Thanks guys!

[Daniel Browning]

Not at the same shutter speed and aperture
as ISO 160.

Agreed. I should have clarified that I meant a different exposure.

You would have to shoot ISO 100 at -.7ev to get the same
speed/aperture combination. I'm willing to sacrifice a whole 1/3 stop
or DR for that. The real world result is that I canhandhold at1/125th
rather than 1/80th shutter speedand geta cleaner and possibly
sharperimage.

Me too. I only use ISO 100 when I have plenty of exposure to spare. (1/3 stop is such a small difference anyway.)

Which 'tweener ISOs are less bad and which ones should be avoided?

The "minus 1/3" ISO (160 320 640 1250) are fine, the "plus 1/3" ISO are suboptimal (125 250 500 1000). Everything above ISO 1600 is suboptimal.

One more question - is any of this relevant to amateur photography?

It's a pretty minor effect by itself. Just like HTP is a pretty small effect, and ALO is a pretty small effect. But when you start combining all these small factors, it results in a big factor. For example, ISO 250 by itself may not increase shadow noise enough to notice. And ISO 200+HTP by itself may not either. But combine 250+HTP and the result will be much more noticeable: the shadow noise is as bad as ISO 640. Add ALO into the mix and you can get people wondering why their ISO 250 shot looks like ISO 1600.