I don't think you have anything to worry about using a hand hammer, given your description. That is provided the anvil has no weak spots (casting flaw or cracks).
The only old anvil I have seen get the heel broken off was someone using the hardy hole to drive a hardy with a large stem into the hole to size it. If I remember he was using an 8 pound hammer and the heat had just about gone out of the stock. I blamed that on operator error, he should have forged the stem down closer to the right size.
I have done some pretty heavy hammering on our MP cast steel Farrier's anvil's heel along with the 110 pound Vulcan, using my go to 2 1/2 pound cross peen hammer and they are still good to go.