The three lenses you are talking about, in order of image quality - EF 70-200mm f/4L USM is better than EF 70-300mm f/4-5.6 IS USM is quite a bit better than EF-S 55-250mm f/4-5.6 IS.


Jan's suggestion to turn off IS on your 55-250mm lens is a great one - if you can live without the IS, the EF 70-200mm f/4L will give you the best image quality of the three. You won't miss the 55-70mm region, and in comparing the 70-200 with the 70-300, keep in mind that the optically worst region of the 70-300mm is the 200-300mm range.