Red Highlights

[Dumien]

I usually take pictures of flowers in my back yard because my mom loves them. I usually find myself dealinf with a problem in the red channel: more often than not, I encounter a problem with saturation of the reds in the highlights. I usually shoot in the morning or in the evening, because the light is less harsh and always use highlight tone priority.


Here's an example, first the whole picture, then a crop of an area involved in the problem.


canon XSi, 24-70mm, 1/320sec, F5.6, ISO 100, 57mm


http://img6.imageshack.us/img6/2025/dpp0009j.jpg

http://img197.imageshack.us/img197/9343/dpp00099.jpg


Usually, some saturation and curve adjusting is enough to get rid of most of these phenomena. BUT, what would you suggest to prevent this kind of things happening?

[alex]

any chance you're shooting with the Landscape setting, instead of Standard? I find that the Landscape setting on my XSi often over-saturates the colors.


I didn't know the XSi had Highlight Tone Priority. I need to look at my manual again, I guess.

[john-pa]

I shoot a lot of flowers too, and I often see this effect. One of my early theories was that the IR wavelengths were fooling the camera’s metering, and I tried using an IR-blocking filter, but that did nothing. These days, when faced with highly saturated reds, I just make a point of doing at least -1ev, and that usually solves the problem.

[Daniel Browning]

It looks like you're referring to the blown reds. Many times the red channel in the raw file itself is fine, but gets blown during raw conversion: especially applying white balance or during color space conversion (i.e. sRGB). It appears that you used DPP, which can exhibit this problem too (although I recall it has pre-WB linear EC, so you could work around the first problem by applying a nonlinear EC curve in Adobe after conversion).

I suggest posting the raw file (e.g. yousendit.com), then one of us on the forum can show you what it looks like in a different converter (or in DPP with different conversion parameters and post processing).

[Dumien]

any chance you're shooting with the Landscape setting, instead of Standard? I find that the Landscape setting on my XSi often over-saturates the colors.


I didn't know the XSi had Highlight Tone Priority. I need to look at my manual again, I guess.
<div style="clear: both;"]</div>





Yes I almost always shoot in Landscape setting, because I like how it renders colors, but I usually try all of the settings out when I'm post-processing. This one was converted using the picture style "faithful"


Daniel,
can I send it to you? Could you give me your email? Thanks


Andy

[Daniel Browning]

Andy,


I sent you my e-mail address(es) in a private message ("conversation") using the forum, but I haven't heard back. Just in case you didn't get it, here it is: db (at) kavod (dot) com. (What's another 100 spams per day? I already filter thousands.)

[Don Burkett]

Reds, purples, yellows, oranges, are all difficult colors to render perfectly. They are so saturated in color it's easy for them to blow out. Most especially anytime they are in direct sunlight. My suggestions are spot meter then go to manual mode, start with an exposure compensation of -1 and keep heading south until you get the accurate color. If you get to -2, use a diffuser and start over. By the way, when these same colors are in full shadow, getting accurate color is still an experimental process, partitcularly in the deeper tones. In short, in most cases do not rely on the camera's default settings when shooting these colors when their fully saturated.

[Daniel Browning]

After Andy got back to his computer he sent me the raw file. Here it is for anyone who would like to take a crack at it:


IMG_0019.CR2 ("http://thebrownings.name/images/blown-reds/IMG_0019.CR2)


I took a quick glance and it is just as I expected: the raw file itself has absolutely *no* clipping in it anywhere. (I checked with Rawnalyze.)


In fact, it is *underexposed* by 2 and 1/3 stop! Yet when I open the file in DPP, the reds are immediately clipped. This is a very clear case of a raw conversion gone horribly wrong. :) The good news is that it can be fixed.


The white balance settings include "push RED channel by 2.29 stops", so most of the clipping is probably happening during the application of white balance. I will see if I can get DPP to do a more sane conversion. If not, I'll show what it looks like in a different (better) converter.

[Dumien]

maybe this might help:


in DPP I upped the exposure by 0.33 and then applied kelvin white balance to 4600 degrees. neither contrast nor saturation were applied. Raw sahrpness to 7 and RGB sharpness to 80...


I think that's all I did ^^


thanks for the help


Andy

[Daniel Browning]

Keep in mind that I'm no post-processing expert. I can only prove that better processing is *possible*, not actually do it myself. :)


I couldn't get the results I was looking for in DPP (I don't use it much), so I did quick run with RPP:


http://thebrownings.name/images/blown-reds/IMG_0019-preview.jpeg ("http://thebrownings.name/images/blown-reds/IMG_0019.jpeg)


(Full size image ("http://thebrownings.name/images/blown-reds/IMG_0019.jpeg))


I'm sure an expert could do much better. The color is obviously very different from yours. But at least it proves that the issue is entirely contained to raw conversion and post processing. It's not a camera or exposure issue. In fact, the shot could be exposed 2.3 stops hotter without blowing any highlights. (In this case that's not important since the image clearly has sufficient DR and noise, but it helps to know that for other cases that do need more DR and less shadow noise.)


Going back to your original question: the best way to avoid this problem is to wait for technology to come up with computer displays that are capable of reproducing the same colors that occur in real life. Of course, they will probably invent teleporters and space travel first, so in the mean time we have to work with software to try and find ways of mapping those extremely saturated colors down to the limited level that monitors are capable of. But since I haven't really learned enough about this stuff, the best answer I can give you is probably "I don't know."

[Mark Elberson]

Daniel


What do you use for RAW conversion?

[Daniel Browning]

Daniel

What do you use for RAW conversion?


Mostly Lightroom. It has a ton of quality problems. Underexposed (high ISO) shots are especially poor. It lacks basic features such as nonlinear EC. The downsampling algorithm corrupts the image with aliasing artifacts. It applies different amounts of hidden exposure compensation for different cameras (e.g. you have to set -0.33 EC for the 5D2 to get counteract their hidden +0.33). There are plenty of other quality issues as well.

But for all its problems, I still prefer it over all the others for one single reason: ease of use. I can get on the computer, process hundreds of photos with various adjustments here and there, and get back off in a short amount of time. The plugins and integration with other software save more time still. Most of my photos look "good enough" with Lightroom, so I put up with the quality issues just to be able to zip through a pile of raws in less time. Before Lightroom 2 came out, I was using Bibble for the same reason: easy. But it took extra time to integrate with the DAM. Lightroom has the DAM integrated directly, which I think saves me some time (at a loss in flexibility).

When I am concerned about quality, I will take the time to load up a different converter. My favorites are RPP, DxO, but sometimes I try Bibble, DPP, or a dcraw derivative. I develop astro shots in a variety of more specialized programs such as IRIS, DSS, and MaxIm DL. I haven't used C1P, Aperture, or SilkyPix, but I hear good things.

[Mark Elberson]

I use DPP and find it very user friendly and with its"recipes" and batch processing I think it can bang out a lot of images in a reasonable amount of time as well. I am curious though what your thoughts are on its quality?

[Daniel Browning]

I think the quality of DPP is a sight better than Lightroom, especially for underexposed (high ISO) images. I really like that NR=off really means "off". (Whereas in Lightroom, NR=off means "a little less".) I also really like time-saving features like automatic falloff correction. However, I think it introduces a more demosaic artifacts (e.g. maze) than other converters, but they're not noticeable in most shots. One thing that really annoys me about DPP is that it doesn't implement 1/3rd stop ISO correctly. Those ISO settings have different white points (i.e. "highest number possible") than normal ISO, but DPP uses the same white point for every ISO setting, so it completely misses the very tippy top of the highlights in 1/3rd stop ISOs (whole stop ISOs are fine). It's only a small amount of highlights, but just the idea of wastage just bothers me. (You'd think that if they're going to mess up the camera by changing the white point for each ISO setting that they would at least handle it in their own converter correctly, but oh well.)


Also, as demonstrated in this thread, I can't figure out how to control clipping caused by color space conversion in DPP, but there's probably a way.

[Dumien]

Well Daniel, what can I say? Your insights into this kind of technology are very well appreciated.


Thanks you all guys for helping me out


Andy

[Chuck Lee]

Andy,


I'm a bit late to the party but I had to wait until I was home to do this.


I didn't see the 2.3EV worth of headroom on this raw. I used three raw converters andnone neededan EV adjustment. I was very suprised to see how good the exposure was. I'm sure with the first two you could easily tweak things to get the color more like you want it but I thought the default renderings were pretty interesting. Here goes.


Warning: The followingimagesmay exceed the color gamut ofsome monitors. There are shades that I cannot view on my laptop lcd. On my HP LP2275w each of the following images looks great. Even your original posted image looks pretty clean!


SilkyPix Development Studio Ver 3.0.25.1 - Default Settings


http://www.partsense.com/Photos/RedHighlights/IMG_0019_SP-Default_640B.jpg


Original SilkyPix Here ("http://www.partsense.com/Photos/RedHighlights/IMG_0019_SP-Default.jpg)


ACR 4.4 with ACR4.4 selected as the camera profile. Default Settings


http://www.partsense.com/Photos/RedHighlights/IMG_0019_ACR4.4_-.5ev_640.jpg


Original ACR Here ("http://www.partsense.com/Photos/RedHighlights/IMG_0019_ACR4.4_-.5ev.jpg)


Picasa from Google - Go figure, I like the way it rendered the 450D RAW!


http://www.partsense.com/Photos/RedHighlights/IMG_0019_Picasa_640.jpg


Original Picasa Here ("http://www.partsense.com/Photos/RedHighlights/IMG_0019_Picasa.jpg)


I don't like and don't use DPP for obvious reasons. I don't think that's necessarily the problem. It may have alot to do with the extreme color gamuts of the photo being displayed on a monitor that doesn't have the ability to display those gradients properly. I don't know what kind of printer you have but I'd try printing it and see what you get. You may be suprised.


Out of curiosity I am interested to know which rendering is the closest to what you were looking for. (If any were close)


Chuck

[Daniel Browning]

Great conversions, Chuck. I like the colors on the Picasa (a dcraw-derivative) one too.



I didn't see the 2.3EV worth of headroom on this raw.


I'm not surprised. Few photographers have ever even seen a raw histogram; only what that has been processed (and mangled) by their converter (or camera). That's why I used Rawnalyze: it lets one see the *true* histogram, before white balance, black clip, gamma, tone curve, etc.


(That's reason why I shoot UniWB, which allows me to see the true histogram in-camera and get +2 or +2.6 more exposure, which means a ton less noise without clipping, among other benefits.)



I used three raw converters andnone neededan EV adjustment.


The optimal exposure is totally disconnected from the amount of EV adjustment needed to achieve a pleasing brightness. Generally, the optimal exposure is "as much as possible without clipping highlights" (ETTR). For this image, the exposure could have been increased 2.3 stops without clipping. Then in post production, one reduces exposure by -2.3 EV to get the brightness back to the desired level. (But since raw converters never show the raw histogram, no one ever realizes that they have so much headroom.)

[Colin]

Ah, so are you saying that, in terms of dynamic range and detail, it's generally better to overexpose and then dial back in the raw conversion?


One question on the raw conversion....


For web display, i'd assume that we'd be looking at sRGB or what have you. If that's the case, shouldn't all of these images look similar? Or were we supposed to download them and then view them not in the web browser, or do some web browsers support embedded color space information?

[Chuck Lee]

Daniel,


Thanks for the pat. I did learn something today. I was glad I waited until I got home to look at some things before I posted. I was amazed to see those images on my new HP monitor. Obviously, we're all not seeing the same thing when it comes to high gamut renderings like these red flowers. On my HP2275w there is no visible clipping that I can see in the OP's image.


So Picasa uses a dcraw derivative for thier engine. That's interesting. I've heard that name many times. Can't say I've used it, but It's got quite a reputation along with RawTherapee (I think that's it).


Any way, I'm a SilkyPix diehard. It just flat out works for me 99% of the time. It's nice to see the differences with ACR and something as simple as Picasa from time to time. Helps keep my SP output honest and serves as a good scale for hard to render photos like this one.


Colin, Each RAW converter will render the RAW data slightly different from one another. I'll let Daniel do the in-depth. The originals are for anyone who wants to check the gradients up close and personal. If you really want to see something wild, develop this CR2 with ACR 4.4 or higher along with the Canon camera profiles. Standard, Faithful, Neutral, Landscape, etc. The outputs are miles apart when pushing a red gamut this hard. sRGB is as good as all the other color spaces when developing for the web.


That's why I'm curious as to which output renders the most true. I'd like to think that SilkyPix does. At least for me, 90% of the time it's really faithful at rendering accurate color output.

[Chuck Lee]

(That's reason why I shoot UniWB, which allows me to see the true histogram in-camera and get +2 or +2.6 more exposure, which means a ton less noise without clipping, among other benefits.)


Daniel,


http://photography-on-the.net/forum/showthread.php?t=485349 ("http://photography-on-the.net/forum/showthread.php?t=485349)


Is there any validity to this one. Like that poster, I'm lazy too.


If your taking about the green cast white balance for accurate raw histograms, I tried, but just couldn't get use to it. Most of the time, under normal conditions I shoot 40D at +.3ev and the 5D no-comp. I use center weighted alot but find that evaluative works best most of the time on my 5D. I try to set custom WB based on conditions and shoot with "K" selected WB. I find that 5000K is a great place to park!! I try to shoot "to the right" in high ISO and that works pretty good.


The biggest thing that I keep messing up on is forgetting to shoot a white/gray card at the beginning of session. Man, I could save myself hours of post time if I'd just remember that one silly little thing. If I could get in the habit I'd spend 70.00 on a Gretag color chart and use that as well!


And this is good advise to the OP to practice doing when shooting flowers, etc. I suspect 4600 Kelvin is just a bit cool for that scene. A white/gray card could have really helped there. We would all have a better idea of what color those red flowers really are!!

[Colin]

Daniel,


Colin, Each RAW converter will render the RAW data slightly different from one another. I'll let Daniel do the in-depth. The originals are for anyone who wants to check the gradients up close and personal. If you really want to see something wild, develop this CR2 with ACR 4.4 or higher along with the Canon camera profiles. Standard, Faithful, Neutral, Landscape, etc. The outputs are miles apart when pushing a red gamut this hard. sRGB is as good as all the other color spaces when developing for the web.


That's why I'm curious as to which output renders the most true. I'd like to think that SilkyPix does. At least for me, 90% of the time it's really faithful at rendering accurate color output.
<div style="CLEAR: both"]</div>



My only experience in rendering raw data is with DPP and Adobe Lightroom.


According to the Canon descriptions, Faithful is the only rendering option that is accurate. Neutral is compressed in color space to minimize clipping and expects generous post-processing. Everything else is a stock enhancement preset. I played with lightroom for awhile, but while I subjectively liked a few renderings better in their effect, I couldn't get images that looked much like the 'Faithful' initial settings on DPP.


Something, though, bothers me. While I'm all for tweaking images for the best representation of what we want to present, if different software takes the same data and, by default, presents a drastically different image in the same color space, then, technically speaking, I would think that means that they can't all be right, i.e., most, if not all of them, are wrong.


I think I'll put these more advanced raw rending methods on my long term wish list, along with Photoshop and more sophisticated noise reduction software. Need more time for this.... [:P]

[Daniel Browning]

Ah, so are you saying that, in terms of dynamic range and detail, it's generally better to overexpose and then dial back in the raw conversion?


Yes. When most folks do "ETTR", they don't realize that they are only doing ETTR for the JPEG on their camera, which is easily two stops off from the raw data (as in this case).



For web display, i'd assume that we'd be looking at sRGB or what have you.

Correct.



If that's the case, shouldn't all of these images look similar?


As Chuck said, the differences are due to different converters and different settings. Colors that can't be displayed in sRGB (such as the true saturated red of this flower captured by the raw file) can still be *mapped* to a similar color in sRGB. (Just as you can't take a picture of the sun and print it on paper that is bright enough to blind people: you have to "map" the brightness of the sun to a lower brightness on the paper because the paper just isn't capable of reproducing the natural world.)



Thanks for the pat. I did learn something today. I was glad I waited until I got home to look at some things before I posted. I was amazed to see those images on my new HP monitor. Obviously, we're all not seeing the same thing when it comes to high gamut renderings like these red flowers.


Yes, cheaper monitors and laptops (like my Macbook Pro) do an especially poor job. I'm using a NEC 2490WUXi, which has very good reproduction of sRGB.



On my HP2275w there is no visible clipping that I can see in the OP's image.


Pull it into photoshop and isolate the red channel to see the clipping. There are many areas with no detail (253+). The blue and green channels are fine, which is why we can still see some detail in those areas.

Of course, most monitors will show even more of it as blown (including blue and green channels) even though it isn't.



sRGB is as good as all the other color spaces when developing for the web.


I would add that most computers/browsers aren't color aware and so cannot deal with anything other than sRGB. But it would be possible to upload a 16-bit tiff that has a color space with a very wide gamut (e.g. BetaRGB) so that nothing was clipped in the image *file*. But to view it, the display has to convert it to sRGB anyway, and so it will get clipped. So unless it will be printed, it's better to do a quality conversion to sRGB (and avoid clipped colors) during raw conversion.




Daniel,

http://photography-on-the.net/forum/showthread.php?t=485349

Is there any validity to this one. Like that poster, I'm lazy too.


Yes. Those instructions will often get your Uni-WB within 1/3 stop, which is close enough for most folks. The best method takes a little more work, but you only have to do it once in the life of the camera:

http://www.dpreview.com/forums/read.asp?forum=1021&amp;message=31582853

One of these days I'm going to try to build a Custom Picture Profile that does a reverse sRGB curve like so:

http://www.dpreview.com/forums/read.asp?forum=1039&amp;message=31842306

That would allow the histogram to be linear instead of gamma, which is much easier to see and use.



If your taking about the green cast white balance for accurate raw histograms,


Yes, that's precisely it.



I tried, but just couldn't get use to it.


The way I do it is to switch back and forth, because color is obviously an important part of composition.



The biggest thing that I keep messing up on is forgetting to shoot a white/gray card at the beginning of session.


Just make sure all your subjects wear something white. ("I hereby award you this commemorative white ribbon in honor of UniWB Awareness Day.")



We would all have a better idea of what color those red flowers really are!!


Indeed. One of those mysteries that keeps our interest. ("Were those skies really that purple? Or was it just Velvia film? We'll never know.")


Something, though, bothers me. While I'm all for tweaking images for the best representation of what we want to present, if different software takes the same data and, by default, presents a drastically different image in the same color space, then, technically speaking, I would think that means that they can't all be right, i.e., most, if not all of them, are wrong.

That's certainly a valid way to look at it. Technically, they're all wrong. All that matters is the one that gets in wrong in the most pleasing way. :)

[Chuck Lee]

Colin,


I personally like SilkyPix. I started using it when I shot Pentax.I'm use to it and can get alot out of my photos. Most of the time I can pull around 1 to 2 ev out of blown highlight land. I don't shoot for that, but can use it if it's there. I find most photographers that shoot in raw find a particular program they like andworks for them.Raw rendering color accuracy is a subjective game that requires a bit of practice with ones chosen program. Once a confidence level is reached, it is very hard to switch to something else. Look at Bryan's photos. His color is as consistent across all images as any photographer I've ever seen. It is more than obvious that he knows "his" workflow. Whether, raw or in-camera, if you know what the output is going to look like based on the control variables you can get repeatable results and feel confident with what you are doing. Heck, just try shooting the same scene in JPEGusing the in camera scene modifiers; Standard, Faithful, Landscape, etc. and you will see all the variationthat the built-in raw processor can offer. Which one of those is right?


So, Take your time.......it's alot to digest.


Later...

[Dumien]

SilkyPix Development Studio Ver 3.0.25.1 - Default Settings


This one looks kind of purple on my monitor, or not really red, whatever color that may be.



ACR 4.4 with ACR4.4 selected as the camera profile. Default Settings


This one is the one I prefer, it's very nice, thanks :D



Picasa from Google - Go figure, I like the way it rendered the 450D RAW!


As much as I think it's weird that PIcasa did something like that, I think it's pretty good, but not as good as the ACR one...





still, my monitor sucks, i guess ... lol


thanks again,


Andy

[Chuck Lee]

Yea, I had a sneaky suspicion ACR was the one. I sort of expected that after seeing the Picasa version. Your right, the SilkyPix default was heavy on the magenta. The first thing I would have done was back that off. SP has a tendency to use a fairly heavy color deflection when dealing with Canon raws. It is usally toward the magenta side. But, that's OK, I think I would have gotten it close just processing it the way I do most of my shots. I am always curious to know how the "default" renders. I forgot to add that Irfanview can also be used to convert CR2 files for those that like free stuff.


I didn't know what I was missing until I bought the HP2275w. Unfortunately, it's defaults are way over the top and it needed to be adjusted. That's a whole other story. I'm sorry you have a "sucky" monitor. If you decide to upgrade, do good research. The LCD stuff today is very tricky.


Thanks Andy, Great thread!


Daniel,


I had a "Duh!!" at 7:00AM today. The 2.3 ev headroom you were taking about was extra usable space in the raw histogram. No data was there, so there wasn't any reason to see clipped red highlights. Got it!! That's why I commented to Colin about pulling down 1-2 EV of blown highlight data when using RAW. It all came together with that first cup of coffee. Sorry bout that Daniel. I'm denser some days than others.


Chuck