You just have to use a different approach, Dave. I used to build tools for a living and didn't ever use counterbores for SHCS's. Counterbores are truly a PITA, particularly if you get a chip between the pilot and the hole, or one floating under one of the cutting edges. I used them in production drilling long ago, so I have a fair understanding of their application, and the pitfalls of their use.
For SHCS's, using one of my Albrecht drill chucks, I start my c'bore with a twist drill, the same size as the desired c'bore. I drill deep enough to generate a full diameter, then switch to a flat bottomed drill. The cutting speed (of the drill, as opposed to a c'bore) makes up for the time lost changing the tool, and the pilot developed by the first drill prevents the flat bottom drill from wandering about. You end up with very nice counterbored holes, with no money invested in tooling that is often more trouble than it's worth. All of this, of course, works very best when you're using a drop spindle mill (like your BP), not a drill press. You have far better control over everything that way.
Give it a go! It served me very well for years.