Let me lay a little groundwork first.Originally Posted by Jon Ruyle
In-camera ISO is just gain. It can be digital or analog. If the gain is digital, then all it is doing is mangling and deleting the raw data. One stop of digital gain (ISO) literally deletes one stop of highlights, with absolutely no improvement in read noise or quantization error.
If the gain is analog, it still deletes highlights, but there are two possible benefits: less read noise and/or less quantization error.
Digital gain never does either of those things, so it's always bad to use on raw data: it should only be for JPEG embedded previews. On all well designed cameras, quantization error is not a problem, so that's not a reason. So the only good time to ever use gain is to get less read noise.
The only time you get less read noise on Canon cameras is at ISO 200-1600. ISO 3200 is analog, but the read noise does not improve. ISO 6400-25600 are just digital and never have purpose.
So metadata-based ISO is separate from and in addition to the analog ISO. It's just a recommendation to the raw converter to apply exposure compensation. For example, if you want to shoot an ultra low light portrait at ISO 25600. With Canon's current cameras, you have two choices:
1. Set the ISO to 25600. This causes the camera to use an analog gain of 3200, then apply a digital four stop push, deleting four stops of highlight data, blowing and clipping important details all over the scene, as well as doubling or tripling the file size.
2. Set the ISO to 1600, which is the highest useful analog gain. This is 4 stops away from the desired gain, so the exposure meter, autoexposure system, and LCD review will all be useless. In post-production, nonlinear exposure compensation may be used to apply the gain without deleting the highlights needlessly: saving all four stops of information, and keeping the filesize small as a bonus.
What I would like is a third option:
3. Set the analog ISO to 1600 and the metadata ISO to 25600. This causes a highlight-preserving EC to be used for in-camera JPEG (or even a linear EC) while leaving the raw alone. This is the best of all worlds: it retains the highlight data, file size, AE system, and LCD review.
This feature would also unlock the possibility of getting more highlight headroom even at low ISO. Right now, if you want to shoot ISO 1600 with the highlight headroom of ISO 200, you have to give up auto exposure, metering, LCD preview, etc. and just shoot analog ISO 200. With metadata ISO, you could get all those features back by just setting the metadata ISO to 1600.
So it's not just for super high ISO, it's useful at *anything* over ISO 100 when you want more highlight headroom.
Canon had this feature in the 10D. ISO 1600 was analog ISO 800 with "+1 EC" metadata that was applied to the jpeg preview and in-camera LCD and for autoexposure metering and review, but did not mangle the raw data. All subsequent cameras had this feature removed, and instead began deleting highlights from the raw data.
Years later, Canon added Highlight Tone Priority, but limited it to one stop and disabled it for some of the ISO settings where it would be most useful (12800 and 25600), and added it to the ISO settings where it can cause a lot of noise (especially ISO 200).
If you simulate metadata ISO 25600 by doing a 5 stop push of analog ISO 800, then compare it with Canon's current ISO 25600, it has 5 stops more highlights with almost the same amount of read noise. Simulating 25600 with analog ISO 1600, the read noise is *exactly* (not almost) the same, and the difference is 4 stops of highlights.
The next question on your mind is probably this: If it were so easy for Canon to preserve 5 stops of highlights, why would they delete them? I think there are many possible reasons:
* Because they can. Customers accept all sorts of compromises at high ISO, even ones that are completely unnecessary (like this). Many photographers never even use high ISO. Canon knows that few people use it and the ones who do just have to be willing to accept the problems.
* Because of Customer Service. Some customers will have a rude awakening when they realize that Canon has been deleting 1, 2, or more stops of their highlight headroom for no reason. Even if Canon buries the metadata feature in some custom function, and provides copious documentation, many customers will get thoroughly confused about what it is or does. Misconceptions will travel rapidly on the web about it, no doubt. Even some raw processing programs might get confused by the metadata. Adobe, for example, doesn't implement HTP correctly, it just does a linear push without preserving highlights. Other converters might not apply the push at all, so customers will wonder why it's too dark.
* Because of Management. Canon had ISO metadata in the 10D, but removed it in all later cameras. That makes me think it was removed on purpose, as if by edict from Managament. Software engineers wouldn't remove a feature that was doing the obviously correct thing and replace it with the obviously incorrect thing for no good reason. A good reason would be if the manager said "cripple the camera or I break-a-you-face."
* Because of Marketing. Perhaps the feature was removed because Canon wanted to save that trick for a rainy day, so they could come out with a new camera with "4 stops more dynamic range at high ISO!" without any development cost. If they were nefarious enough, they could sell a firmware update to all cameras going back to the 20D offering ISO 25600 as metadata, with 4 stops more dynamic range accross all cameras. Sony sells $1,000 firmware upgrades on some of its high end video cameras, so there's precedent.
* Because of Engineering. It's possible that the software engineers responsible for the firmware are unaware of the issue, but given the high level of competence they've demonstrated I think that's highly unlikely. I don't think Hanlon's razor applies.
* Because of Bureaucracy. This is the most likely reason, IMHO. Many suboptimal things happen in big corporations. Maybe the 10D metadata ISO feature did not have the correct "addition of feature form" filled out, and the engineer who added it was fired because he didn't follow the correct procedure for requesting authority to add a new feature. Maybe the paperwork for removing the feature was easier than the paperwork required for leaving it in. Who knows.
In any case, after all these years, I have little hope that Canon will make an improvement in this area any time soon.
Other manufacturers, however, including many MFDB and digicams, do already implement ISO as metadata. Of course, they don't have read noise improvements at high ISO read noise, so they don't even *need* the feature like Canon cameras do. The cameras that need it don't have it. The cameras that have it don't need it. Funny world.
Most people think "fake" ISO is bad because it's its not amplified in analog. But that's not a problem, in fact, that's the most desirable state. To have a camera with such high bit depth and such low noise amplification that noise is the same at all amplification levels would yeild the highest amount of dynamic range. So in that sense, "fake" ISO is the best ISO. The problem is that the only "fake" ISO implementations that Canon uses (except once, in the Canon 10D) are done by mangling and deleting data in the raw file before it's written. It's much better to just write metadata saying "delete 5 stops of highlights because every photographer loves blown highlights", then some would get the file into post processing and change that little recommendation.
When one shoots wider than f/2.8, Canon mangles the raw data with a digital push in order to compensate for lower sensor response to light from oblique angles due to the microlenses. You can read about where it's done in the 30D here, but Canon didn't fix it for the 5D2, and I doubt the 50D or 60D will be corrected either:
http://www.pages.drexel.edu/~par24/rawhistogram/CanonRawScaling/CanonRawScaling.html
The reason they do this is for all the same reasons as above. Papering over the sensor angle of response with a digital push helps Customer Service issues (people don't wonder why their f/1.2 is darker than f/2.8), Marketing, Management, Beurocracy, etc. Of course, if you slightly untwist the lens so that there is no electronic communication, then the camera will not know to apply the push to your f/1.2 or f/1.4 lens, which allows you to get the unmodified raw data. There's really no good reason to mangle the raw data IMHO.
That would be really great. CHDK is a stellar example of what can be done even when Canon is fighting to prevent it. And here I would have thought a legion of unpaid programmers adding tremendous new features would be a good thing.
If Canon open sources their firmware, I will add the metadata ISO feature myself.
Reply With Quote