Nope!
You only need to eat the skin if you're using it as a Hex B on a red day, because, like you said, that's where the fibre is.
On Green and EE, it's free either way - otherwise, mashed potatoes and SW chips, for example, wouldn't be free on plan either!!