I have the 35/1.4L and the 40/2.8 'pancake' and both make good 'walkaround' primes. The 35L allows more creativity, and on a FF camera, the AoV is wide enough that as long as you're not on top of your subject(s), the DoF even wide open is not razor-thin - thus, the amount of light let in at f/1.4 - f/2 makes it a great nighttime walkaround lens, and that's often how I use it. I got my 35L a couple of years ago...if I had to make the decision today between the 35L and the new Sigma 35/1.4, the Sigma might be my first 3rd party lens...

The 40/2.8 is a perfect 'extra' lens. I bring it when my primary lens is a white zoom, either the 70-200 II or 100-400. The 40/2.8 fits in a pocket, ready if I need it. It also doubles as a body cap.