One suggestion. Look through the photos you have taken. If a lot of the ones where you wish you had a wider aperture are in the vicinity of a prime, that might be the way to go. The big advantage of primes is the wider aperture and that most are better in the IQ department than the zooms.

17-55 and 24-105 is a lot of overlap that costs several thousand dollars to achieve! If you do it, I'd suspect the 17-55 will be used indoors and the 24-105 outdoors. This just may be the right combination for you. I carry the 24-105 on a FF almost everywhere I go. On my crop body it covers the equivalent of around 40-180 mm focal lengths. If you don't do landscapes and do lots of portraits the range is great but the f4 DOF isn't going to be a strongpoint. When I head off to do a shoot, the 24-105 is usually left behind as I have other lenses better suited to most of the work I do. It's a great general purpose lens, but I'd like it even more if it had a wider aperture for action and portrait work.

Primes or really good zooms is a choice most people eventually make. Either of the two zooms are good lenses, but the questions you need to ask yourself is whether the aperture, wider angle, or reach is most important to you? Or do your existing photos tell you primes are the answer?