I love my 70-200 f/4 IS. I think I'd go that + 1.4 extender (Gen II works better for this old lens) rather than buying the 100-400. That's a lot of IQ bang-for-the-buck I know a lot of people swear by their 100-400 but I just didn't care for the one I rented at all. The 85mm lenses are quite nice (I have a 1.2 now, and a 50/1.4) but honestly, I tend to hit that focal length with my 70-200 as often or more than I grab the prime. Don't discount IS at the pixel level even at "fast" shutter speeds (see the D800 thread to beat this horse dead). Obviously the prime is faster but the IQ of the 70-200 f/4 IS is really, really good at normal portrait apertures (5.6+).

At the short end I only have a kit lens but will be buying EF-S 17-55 next chance I get.