I do. You'd be surprised how quickly they add up. One banana = 100 calories. Some onion, carrot, mushrooms, pepper etc in dinner can add up to 150-200 calories.
On there own it may not seem like a lot but when you add them up over the course of a day they quickly contribute alot of calories. If you're creating a deficit of 500 calories per day (1lb per week weight loss) it can really affect your weight loss.
So count every single thing that you eat (and drink).