Depends what plan you're on really. I'm doing All About W8 and we're allowed half a pint of skimmed milk per day for our tea/coffee and it hasn't put me out of ketosis. I've lost 20lbs and 5 waist inches in 4 weeks, and that's a good enough result for me. Who knows if I would have lost more, but being able to have teas and coffees throughout the day makes things more manageable.
Val x