Yes, it's an interesting question. I am tempted to go by weight....because it seems truly measurable. (unlike sizes that vary so much from shop to shop....I feel like I'll never really know whether I'm a size ten/twelve/fourteen or not anyway! I'd always be waiting for one shop/outfit to catch me out and make me feel rubbish again)
On the other hand, as you say, sometimes, some people's weight is surprising. My consultant always says she's surprised I weight what I do, I don't look it etc. I think I do weigh on the heavy side as do my sisters. So when I read people's before and after stories in magazines (I love those stories and pics!) I am sometimes so shocked with how little they weigh...in both the before and after pics!!! Sometimes they weigh before, what I would like to weigh AFTER!!! Even if they're the same height as me...It's all so confusing!!!
If only we just KNEW when we look GREAT and at our best...unfortunately this mechanism is a bit messed up in most people....hence why we compare ourselves to others even if it's totally pointless!