If I understand your description correctly, I think there's a way to simplify the process.
This would be to create a sketch as a collection of location points to be used in developing a Sketch Driven Pattern.
When the pattern is defined you can use a single pre-existing Hole Feature (Cut-Extrude, Simple Hole or Hole Wizard type) to automatically populate it with the copied hole axes centered on the point locations.
If all 50-100 holes are not the same, then you'd need a separate sketch for each Sketch Driven Pattern definition.
Just as in the "reference" sketch you described, there would need to be an entity to use in keeping track of the scaling. There's no need to use any in-context conversion of a hole diameter - a construction circle dimensioned to be a starting unit of "1" (inch or mm, etc.) will do.
After the Sketch Driven Pattern is defined, simply use the Tools/Sketch Tools/Modify function to execute a scale change. The original reference circle of "1" unit will become .5, if the scale is made to be one half.
After the first scaling, the reference circle will report a dimension of ".5" unit and, if the sketch is scaled again, changing it back to full size ("1" unit) would of course require a scale factor of 2 and setting it to be 1/4 of the original "1" unit would require a scale factor of 1/2.
Keep in mind that, in order to scale a sketch, there can be no external references. This means that it may be necessary to fully dimension the point locations (instead of using relations) and to delete the X and Y dimensions which locate the entire pattern of points. After the scaling is applied, the X, Y dimensions to the origin of the point patttern can be reapplied.
Hope this helps...
Per O. Hoel