Thank you all for your help so far
I'm leaning toward the #5 right now because its less than 2/3 the price of the #6.
Would the #5 at least do significantly more work than a 4lb hammer at the anvil?
99% of the work I think I will do with it will be pretty light stuff. like forging 3/8" tenons, fullering, butchering, punching through 1/2" bar, chiseling lines, and splitting 1/2" bars.
I'm not going to be selling hammers so it doesn't need to be super efficient at Punching eyes, but it would be nice if it had that capability, and could do it with more ease than I can do it at the anvil. I would only make a hammer once in a while just for myself.
Here is some more good videos I found for anyone else interested in this topic
These are videos of a 5 ton press at work
In this one he forges a small hammer, but it doesn't show him punching the eye so maybe he did that part at the anvil.