Hi Azur,
A couple of things I can see right off. Others may have more and better suggestions.
The fan box would need to be sealed, you're losing pressure out the sides, also a closer fit with wider blades to move more air/higher volume in the space provided. A breast drill also is only giving a low revolutions, gearing to increase revolutions per crank would be more efficient.
If using press board as the building material why not try something along the lines of a box bellows like the Japanese use? Seal it good and make sure it works on the push and pull should get constant air flow.
It's good you are doing this. Keep us posted as to how things are going, and what you decide and do.