Steve
There are many reasons why you might not be getting a good finish on your work - the mechanical condition of your lathe, the type and quality of material being used, the way both the material and the cutting tool is held, choice and condition of tool, as well as the speed and feed employed.
Just as importantly, bear in mind that, as the diameter of the turned work decreases, it becomes progressively more important to ensure that the cutting tip is on the exact centre-line axis of the work and that rigidity plays an ever increasing part in the final surface finish.
It's usual to adjust the angle of knife tool to increase the length of cutting edge in contact with the work for the removal of the last few thou if you want the best finish - give the tool a few strokes with a diamond hone or a stone to get the cutting edge razor sharp to finish.
HTH --
Chris Edwards (in deepest Dorset) "....there *must* be an easier way!"