Although the ideal is to stick to the plan 100%, I think it's better to have a little splash of skimmed milk in coffee and tea rather than suffer abject misery. I don't think it can do that much damage really.
I can manage black coffee but not tea so that's off the menu for a while.