Last fall, when a friend found our I had taken up blacksmithing, he said the same thing. "You look like a blacksmith. I'm not suprised."
I wondered exactly what in tarnation a blacksmith "looks" like. At the time, I hadn't trimmed my beard in over 13 months and it was down to my mid chest. I think he meant the big burly look. But you don't have to look far to see that that's not a real accurate stereotype.
I had this exact topic under discussion at a recent guild meeting. A fellow member mentioned that the demonstrator wouldn't look like a smith if you met him on the street. Then he came to the conclusion on his own that the statement didn't make sense, because who knows what a blacksmith looks like? Glance around at a guild meeting and you'll see all types: short, squatty, burly (like me), tall and thin, big, little, brawny, puny. All it takes is the ability to use your brain to figure out how to apply heat and pressure to metal, not physical strength, most of the time.