Well, they are there to provide a balance to your diet, so it seems daft not to. I do the EE plan, and usually use my HEXA in the form of milk for cups of tea during the day and my HEXB in cereal, but I have that with Mullerlite so dont use any of my HEXA allowance.
I would if I was you, if you can't stomache milk with your "A" allowance, maybe add cheese to your foods in the day? With your B, I don't have bread anymore or cereal on most days, so use my "B" with 2 Alpen Light Bars, and have one of those mid morning and the other late in the eve watching tv. x
Again, if you want to have the full recommended amount then you should eat both. You don't necessarily have to have the ryvita at the same meal though. You could have it later in the day if you wanted. That said as long as you are getting a good dose of fibre/protein/calcium from the rest of your foods then it won't really matter if you don't have the full amount.