My wife and I went to a sword making experience this week as I thought it'd be a good thing to do together, but also I wanted to get a taste for it as I want to get into knifemaking.

I've just purchased an Anvil, Forge, Hammer, and Tongs, and I want to get a belt sander/grinder too but every single one I see that is even kind of powerful is very expensive and out of my budget right now.

Can anyone offer advice on what to do? Can I get away with a weaker belt grinder, or will I just have to get a bunch of metal files and do it all by hand for now until I have saved up enough for a decent grinder?

It would be helpful if you included your location in your profile.  Since this forum has members from all over the world, an answer that might apply in the US may not apply to someone in Belgium or South Africa.  Next we need to get an idea of what you want to do.  It looks like general knife making based on your post, but if there are other things we need to take into account that could affect the answers you get.  Then we need to know what you are willing/able to spend.  

After that we need to get an idea of what resources you already have or have cheap/easy access to.

If you are relatively handy with (and have) basic shop tools you can build your own grinder.   If your junk  resource pile has some square tubing around 1.5" to 2" and you have a working TEFC motor that's at least 1 hp you can put something together with only a little more than your own labor and a wheel set.  If you have to buy new steel and a motor that will make a difference of several hundred dollars US.

If you can give us a little more info we'll try to get you pointed in the right direction.

If you do not want to build one yourself, and don't have the funds for a grinder that has at least a 1.5 hp motor, then I'd suggest using files and saving up for a decent grinder.  You will learn a lot by hand filing and that will actually be beneficial to you when you transition to a grinder.  Hand filing will also encourage you forge close to your final shape so you don't have to spend as much time with the files.