I don't eat cheese at all, and I very rarely drink all of my milk as I'm not a massive fan of that either... Is this a problem? Will it affect my losses?