Sounds like you have slight backlash especially if when you reran the test you didnt move in the initial direction before zeroing your callipers.
Also sounds however if cutting forces may be affecting your spindle and flexing it. Try doing an outside circle and see how it comes out, if its spindle then it should be oversize slightly.
Hood