You do need a little bit of the right fats in your diet, which is why the SW plan works. If anything, I would be concerned that you were altering the balance of the SW plan, if the Xenical made you eliminate that little bit of fat that you do consume.
I was prescribed Xenical years ago when I was ttc and not being successful because I was so overweight. It actually seemed to do more harm than good, and slowed my losses, rather than help them.