Just an idea, a higher flue/chimney might help. If it's low, the smoke hasa habit of staying, but 3-4m(~10feet) up in the air their might be a light breeze which will help carry it away.
Also, after 9am and before 7pm it's generally ok to make as much noise as you want, but within reason(ie. 1-2minutes with an angle grinder, not 1 hour non stop)