To me it looks like the vise might have been slightly unstable on the swivel base when you milled it. I have cleaned up import milling vises, my anvil, and a PYH end mill sharpener like this:
If the top surface wasn't milled, level it with wedges and take a light pass to remove high spots and see how it cuts. Shiny spots are harder and may be high afterwards. Flip it over and block it up on the just-milled surface. Cut enough off the base to see fresh metal all around. Turn it back upright, locate clamps over the cuts on the bottom so the casting doesn't warp, fly-cut the top. I take no more than 0.005" per pass and often 0.001" for the last one, after stoning the bit.
I tried this on an RF-30 mill-drill once without success, the vertical feed was too sloppy. I had to take the vise home to mill it.
Since I have a surface grinder I next grind the bottom side until the wheel cuts at least most of the area, turn it upright and grind the top to taste.
Previously I filed most of the tool marks off and then finished with SiC paper wrapped around a file or a piece of scrap tooling plate. You can file in the middle of a surface without rounding the edges with a file that curved during hardening, which is common for the cheap ones.
I would clean up the swivel base of your beautiful red vise separately on a faceplate in a lathe.
Jim Wilkins