You just have to be closer. I have always thought my best bird shots came from when I was at or very close to the minimum focal distance and that distance is much closer with the 300mm than the 500mm.
If you ever get a chance to pick up a reasonably priced 500mm f/4L IS USM I would recommend it. Just like the 300mm and its successor the original 500mm was an exceptional lens. The upgrade was almost not worth it, the IQ was almost identical and the one thing you gained was weight. Several of those on ebay starting around $2,500.