I'd go for the new 40mm STM f/2.8 pancake.

In fact I have the 35mm f/2 and I want to sell it and buy the 40mm.

Although if by portraits you mean head-shots, you should really go longer, 85mm-135mm.