Drove me nuts too, but peoples perception of you is as someone heavy, thats how they see it, so just smile sweetly and say, dont worry I will!!
When their perceptions change it will be different, I got that loads, but now, they see the slim me, even though I still have trouble lol!!! So I dont get that any more x