I've left a 70-200mm f/2.8 L IS connected to an XTi (residing in a backpack) for extended periods of time without damage to the camera. Cameras are made to withstand a good amount of stress at that connection point. Of course, there could always be a problem with any part of your camera, but I honestly don't think that behavior will harm it.