I thought it was the z, ignore above advice or suggestion.
Sounds like your machine is loosing steps (or something is loose).
What is your x and Y max velocity, acceleration, and steps per unit.
What feedrate, spindle speed, tool diameter, tool material, tool coating, number of flutes on the tool, depth of cut, material were you cutting when this is occurring?
I would suggest you check the tuning on the machine ... get an excel spreadsheet and type g00 x100 (if metric, say 2/3rds of your axis total travel), and the g00 x0... for 100 lines then copy and paste to a text file, save as a *.tap file and test your x axis.... do the same for the Y, then do a blended one (g00 x100 y100 /n g00 x0 y0)
And check how much your machine is out... now bear in mind that this is without any cutting forces si really you should determine where your axis begin to loose steps with acceleration and velocity then cut both numbers back by about 60% and it should be fine.