Both are excellent, of course. Since you have zooms that cover both focal lengths, check past EXIF or set a zoom to each focal length for a while, and see where the thin DoF would be more useful.
Each lens has one 'issue'. The 50L exhibits focus shift (inherent to the lens design, you'll get backfocus at apertures between f/1.4 and ~f/4 with close subjects). The 85L autofocuses quite slowly (and manual focus is no faster because MF works by driving the AF motor).