The 24-105 is a lens that I thought I would sell off after picking up a 24-70 f2.8 some time ago. Never did sell it off. Keep using it as a single lens solution for light weight walk around and explore with minimal gear. Take many images with it, post them online, and print for clients as well as fine art sales. Not once has anyone commented saying that they really like the image but it just wasn't sharp enough, nor made mention of any of the other flaws of the lens as a detraction from the final image or print.
I see that you are considering another f4 lens as a replacement and are citing image quality as the first item on your list. Is this based on feedback from clients looking at their images or is it perhaps a combination of pixel peeping and reading too many gear reviews that makes you think your gear is not adequate? I totally get the urge to upgrade, having done too much of it in the past, but think it might be worth considering what is creating and feeding that urge. When the driving force is that the gear is limiting your ability to create the image that you want because the aperture is not wide enough you will know what you need to get to commit the image in your mind's eye to print. When the focal length is not long enough to abstract the details cleanly from a cluttered background it is easy to know how to fix that.
I ask the questions and make the points I do as I am not seeing a presented reason to upgrade that is based on creative process but rather technical details that come from gear reviews and advertising. What images will a 24-70 f4 lens allow you to make that a 24-105 f4 is preventing you from creating?