No heresy, just speed. Which doesn't much matter for the prototypes, but does matter for production runs. The trouble with the rotary table from scratch is that it will take many passes -- my guess, about 15 or so to avoid having the cutter grab the work and fling it out of the hold-few seconds to cudowns -- or just dig in and ruin the work. It will take a few seconds each to cut the corners off on the band saw and about 10 seconds to rough out round on the belt sander. Then you have only two or maybe three passes to finish the piece. Sure you can go as you suggest -- but an you guarantee that you won't get impatient and take that next cut just a tad deeper -- I know I always do -- and get in trouble when I do. By the way, it isn't about a "fine face mill." I'm assuming you plan to use a vertical milling machine -- in which case, you want to use an end mill (on the side). About a 1/2" diameter running at about 500 rpm should do it for aluminum. You want to keep the speed fairly high for this.
Let me back off a moment while I finally understand what you want to do. You want to use a small diameter end mill (sort of like a router bit and use that to cut successively deeper and deeper. Not the best way for my way of thinking. For one thing, you may end up breaking a lot of little end mills. They will dig in to the aluminum. THey also have a tendency to screw into the work, gradually making your cut deeper than you intended and then, snap!!
Problem with your question is that there are so many, many, correct answers. You'll have to try a few and see which works best for you. But always keep in mind that what you do for the prototype and what you do for production could be totally different.
Boris