Ok, what am I missing?

Everyone seems to hammer the steel down into the hardie hole to form the stem of a hardie tool. But when the steel cools, the stem shrinks and rattles around in the hole.

Am I missing something? How are you supposed to get the stem to sit snugly in the hardie hole?  

Most hardy holes aren’t even truly square, and a tight fir is not to be expected, some smiths will use a wavy spring shim to take up slack, others have used long stems with slots and wedges. That said typicaly driving a hardy into the hardy hole to make the stim isn’t the best idea, setting the flat to the anvil face is another matter. 

That makes sense Charles

I welded some extra steel onto my stem to get it snug

IFI member @TechnicusJoe has a video about making loose bottom tools fit more tightly by wrapping their stems in duct tape. 

Personally, I’ve found a little slop in the fit to be okay. The only tools I have that are super-snug tend to jam in the hardy hole, and getting them out disrupts the work flow. 

I make mine to fit both ways in both my anvils ( makes one direction fit tighter than the other, and each anvil is 90 out from the other for tight. I like long stems to pop them out if the get wedged...) most of my hardy tools started out as sucker rod knuckles, forge down the treaded end caller square and they drop in ( except large gauge with 1 1/4” flats, those I forge down to fit) and leve me long stems. Hot cuts I use the threaded end up, scaling and grinding take care of most the treads and I forge down the rod end. A lot of my tooling is acualy built off center to move the work closer to the waist of my farriers anvils

Dealing with college  students who can't even get it right if you spray paint the anvil and the hardy for it the same colour; (hey it's a well respected engineering school, what would you expect?),  an extra long stem is pretty much mandatory and I taper the section below the bottom of the hardy so they don't rivet it in place trying to remove it when they have jammed it in the wrong anvil.

That’s another advantage to having only one anvil. 

Heresy, I know. 

Or EDM'ing them to the exact same size...

However it's part of training the next crop of smiths how to deal with the variability of hand made tools.

(I also have a dead soft hammer I "suggest" that some students use---I tell them that one advantage to is is that I won't yell at them as much...)

I've seen some smiths form the stem and then hammer it into the hardy to get the right shape & fit.  Am I to assume this is a bad idea?  The only thing I can think of is the risk of breaking off the anvil heel, but I would think hot metal would not do this.  Maybe someone here can explain this a little bit and even relay some horror stories of why this isn't a good idea.

Depends on how hot the stem, red hardness and how close the fit, it’s more important to drive the hot hardy blank in to make sure it sits flat against the anvil. Many beginning smiths don’t get the stock fully hot, that is heated evenly and at maximum plasticity with out burning the steel. Some steels will be fairly hard even at forging temps and if wedge shaped and driven in with to much anthusiasm will put tremendous stress on the heal revealing any flaws in an area forge welded on classic wrought iron anvils or casting flaws a mormmodern ones. Remember this is a square hole, if the corners are sharp they are a stress riser just looking for a crack.

I forge the stem, straiten and reheat. Then fit with a firm tap or two. No go? Forge some more and repeat. If I want tighter I forge to close and grind to fit. I typicaly heat the blank and seat it to insure it sets flat. 

This is one reason that it's a good idea to have a portable hole/striking anvil to do your bottom tool upsetting.