EF 70-300 IS USM or EF 70-200 f/4L USM or ...?

[EricPvpi]

I am looking for advice comparing a few telephoto lenses. I am a beginner in the DSLR area.

I went with a T3i body so I could spend a little more on lenses. I currently have the 15-85mm f/3.5-5.6 IS USM lens as my general lens and I would like something for outdoors, wildlife, kid activities etc. I am looking in the 70-300 range. This would be mainly shooting without a tripod, so IS is probably a plus. Ideally something in the $500 to $600 range. I am willing to spend a little more for quality over a basic lens and I don’t want to find myself looking to upgrade after taking a few shots. I have been reading a lot of similar comparisons, but I couldn't tell if lack of tripod would change the recommendations, and several of the other comparisons are looking at the higher end zoom lenses.

I have been looking at:
• EF-S 55-250mm f/4-5.6 IS
• EF 70-300mm f/4-5.6 IS USM
• EF 70-200mm f/4L USM

The 55-250 is the cheaper route to go, but my guess is the build and picture quality will leave me wanting to upgrade. The 70-200 f/4L is a little above where I wanted to be in price and I worry about not having IS. But if it shoots better, it is close enough in price. The 70-300 sits in the middle. Really I wish there was something in this range with the quality and features of the 15-85 I have with the better ring USM focus.
I have also thought about looking at used 70-200mm f/4L IS USM. But I don’t know enough to evaluate a used lens and make sure there aren’t issues with it.

Any thoughts here, or should I just go with the 70-300? If I went 70-200mm f/4L USM, is it that much better and would I be ok without the IS?

Thanks,
Eric