To save $, I recommend buy as you need + garage sales + patience.
As for storage, what worked out great for me was to layout/label a row x column grid on plywood for all the taps and drills (e.g., the 1/4" column has holes for NC starting, NC bottoming, NC tap drill, NF starting, NF bottoming, NF tap drill), drill (not thru) holes to match taps and drills, and presto you can drill and tap w/o all the hunting. Well worth the 1 hour of effort. My only mistake was to omit 5/16" on the grid since I always use 1/4" or 3/8" -- then I got a Logan lathe which has 5/16 all over it.
David