In my experience it used to be worse in the good old, pre digital, film days, where the only option for indoors photography was with a flash. My old Canon NewF1, especially with the winder (before I could afford the motor drive), sounded like somewhere between a gun shot and a car refusing to start and the large Metz flash went off like an explosion. Not easy to be anonymous then.
Today everyone is taking pictures with their smart phones, with simulated SLR shutter noise and very often also a flash. If the photographer behave polite and avoid the chair standing and other scene dominating activities, I seldom sense any negative waves. But there are times when you just donīt push the shutter and definitely not fire off a flash, unless you have prior approval from the people you photograph. This includes events like the "I do" in weddings, the baptism ceremony, music events (e.g. solo piano) etc. The improved high ISO performance on the newer SLRs helps a lot and I also find the silent mode on the 5DIII very helpful.
Personally I find spontaneous street photography of strangers in certain situations to be difficult. Like a homeless I photographed in New York, going through a trash bin, where I felt I took advantage of his vulnerability and was trespassing on his privacy. He did not react, but I still felt uncomfortable doing it. And I suspect that quite a few of you would share that feeling. But it is probably more in our heads than anything else.
Where ever I go with my camera and there are people involved, I try to be as invisible and anonymous as possible, work on my positioning, be alert and get the right shot when itīs there.