Collet Size Choices

In the past collets have almost always been predetermined for my by the size of the spindle nose I was using.
Well, I am now setting up a new machine with a small ATC carousel. As
part of the setup package and extra stuff I bought I have set screw tool holders, er16 tool holders, and ER20 tool holders along with a collection of collets both brand new with the machine, and that I have accumulated as needed from older machines.
So, how do I decide what to use for what tool. I use a lot of 1/8 shank stuff from 0.125" down to about 0.026". My instinct is to put all that small stuff in ER 16 tool holders, use the ER-20 for any larger tools that I want to optimize concentricity, and use the set screw holders for all my roughing tools.
So, anyway... am I way off base here?
I also picked up some new chucks and arbors, extension compression taper (er16), and some other specialty stuff for the new machine.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
Add image file
Upload is a website by engineers for engineers. It is not affiliated with any of manufacturers or vendors discussed here. All logos and trade names are the property of their respective owners.