I'm actually in the same situation as the OP, but i'm leaning the other way.
On my 7D, I've got the EFs 15-85 for outdoors, hiking, anything in sun, and especially landscapes (more especially when I can only carry one lens).
For low-light i've got the Samyang 35/1.4, Pancake 40/2.8, 50/1.4, 50/1.8, FL 55/1.2, EF 85/1.8, EF 100/2.0. That may be a bit to carry around all at once, so I don't, I pick 2 or 3 for whatever event to which i'm going, maybe swap lenses a few times, sometimes don't even bother.

The f/2.8 zoom (in my opinion), is the does-most-things-ok-but-nothing-the-best compromise.
The 15-85 has the widest, the longest, the IS. The primes have the fast aperture.

So, the decision is between does-it-all in one lens if you can live with the shortcomings (and it's cheaper than a bucket-o-primes), vs what is essentially two kits that does what it does better than the compromise-zoom.
But in your case, given the price differences (depending on for what you can sell the 15-85), i'd keep the 50/1.4 and invest in a longer fast prime, 85/1.8, 100/2.0, 100/2.8 macro to make up some shortcomings in the length of 'only' 17-50.