Like Thomas said. Unless you are using a pointed punch. Then that would be bad for the face of your anvil.
Also the nut under the boss is just a spacer for when you are riveting the first head of the rivet so that you will have the length you need for the other sides head.
Might help to include some pictures, or to be more descriptive of what you are doing and using to do it.