Pesty my family are exactly the same with the "oh but you can't eat that" stuff! Drives me up the wall!
I don't avoid gatherings, and generally I do OK, but I tend to stick to diet coke and if I have the calories left vodka, I end up feeling miserable if I just drink water all night, because I feel like I stand out.
I think the key for me is to plan ahead what I'll have, especially if I'm going to be eating out, that way I already know what I'm going to have and don't worry about it when I'm out.
And choosing clothes I don't feel like a whale in, but that still look OK makes all the difference to whether I enjoy myself or not too.