Likely as not it could be powdered Tantalum, Alum, or a Gypsum like material.. Asbestos would be the less likely also do we not fit locks to "older fire rated doors" on a daily basis, I know I did for over 35 years and when done I would sweep up, scoop up, vacumn any debris which was all over my forearms by then after using flat power bits, hole saws etc. incidentally I would be good and sweaty by then and my skin pores well open , but hey I'm still here, still here, still here, still here,
Wwwwwwhack ThThThThThThTh Thud !!! Aaah tht's better, what I'm trying to say is I'm still still still - Shit I'm OK..
So don't panic as a few older lockies or safe techs if they ever found asbestos used as the only similar items with asbesto I recall was " fireproof" Ha joke - fire resistant filing or document cabinets