If you plan to use actual Plexiglas, you might want to consider Lexan. Lexan is easier to work, doesn't yellow and doesn't scratch as easily.
I haven't seen it mentioned yet but most industrial suppliers stock jobber drill bits with a 135 degree split point. The point is more blunt that the regular 118 degree one found on most bits and the split point grind prevents the bit from wandering. These bits are not much more than regular drill bits and preform better in a range of materials.
You can also purchase what are commonly known as cobalt bits. These are essentially the same, often have the above mentioned split point but have 5 to 8 % cobalt added to the HSS to make them more wear resistant. These are often used to drill stainless steel which is notoriously tough to machine.
Glen