Yes you definitely should. When you wake up, your metabolism is running very slowly as you've been asleep. Until you eat your first mouthful, you don't wake it up to start burning brightly.
Like you, I didn't used to be a breakfast eater "in my previous life" (mind you, look what weight I got to!!), so I still can't eat first thing. I've resolved the issue by arriving at the office early enough to be able to eat in peace before I'm paid!