Can you heat the workpiece to a cherry red? Make a drift from your 1/4 inch toolbit by tapering to the drill size, then drive it through the workpiece at forging heat. Since no metal is removed, the workpiece will bulge - you can file/machine/grind it off when cold. Dimensions and square corners will be perfect. If the workpiece is brass or aluminum maybe you can do it cold.