I am new to blacksmithing, so this might seem like a real newbie type question, but won't blacksmithing make my hammering arm stronger than my work piece holding arm? I feel like after a while it would get noticeable to the point where my right arm would be visibly bigger than my left arm.

To some extent, but that's true of all of us, but our bodies help (working on side when the other side is injured, it will drastically slow the entropy). Besides, you hold and move stock with the off hand, crank blowers, pump bellows... Many things with your off hand. Heck I got tennis elbo in my off arm, lol. 

Good question, and welcome to the your new addiction! I thought about this same thing when I first started a couple years ago, and I must say my right arm gets a good work out swinging the hammer, but my left is right there with it.  Long or hefty pieces of steel will give your non hammer arm a surprising work out. I did a hammer not long ago out of 1.75" 4140 , hot cut it off a longer peice and my holding arm was more tired from manipulating the steel than my right swinging a 6#er.

It is good discipline to hammer with your off hand sometimes!  I have been told that my stronger right side is a problem... affecting proper balance of forces along my spine, among other things.  I do try to mix it up a little... but control is so much better on my right side!  I doubt that you'll find it as much of a problem as you might expect!