I like the way you think... :-)
If he ever brings up the subject again, I thought about saying, "Oh, no problem. I told you the shoot would be free. However, you do owe me for the setup and breakdown that resulted from your inconsiderate lack of communication from the last shoot. That'll be $150."
After he sputters and shakes his head, I won't ever have to deal with him again. :-)