You seem to be making heavy weather of this.
Why are you trying to do both axis at once !!
Backlash is a simple problem - put your cross slide, or carriage in it's most used position. and zero all DRO's.
Ensure backlash compensation is turned OFF
Type in G0 X1. Fix your measuring instrument to measure the position of the carriage and zero it. I use digital calipers. Type in G0 X2 and check the carriage has moved 1 inch - it should have if your pulses per unit are correct. Type in G0 X1 - move the carriage back one inch and check the readout. It should be 0, but will not be - this is the amount of backlash.
Repeat for the other axis.
You can, if you want, do this several times and take an average.
Enter the backlash figure in the table and turn on backlash compensation. Run the test again, and the carriage should return back to zero.If it does not - and say you get a reading of 2 thou - you can add this figure to the backlash compensation, until you find zero.
YOU MUST MOVE THE TABLE TO THE LEFT and then move it right before setting your zero position after each run (to eliminate backlash)
If you are using millimeters then chose a convenient number of milimeters, say 30 - remember, however - the back lash will still read directly, you do not need to divide by 30.
Backlash has nothing to do with the distance the table is travelling. It is only the movement (or NON-movement) of the table, whilst the gears, leadscrew and ball nut settle to the job of pulling the other way.
If you are having trouble because your table is not moving the correct distance, this is all to do with pulses per unit - not backlash.
In measuring distance, you must ensure your table always moves the same way - i.e. to the right - stop - to the right again - stop - measure - until you get pulses per unit and backlash correctly set.
You do not need to move the table great distances to set backlash and pulses per unit. If they move accurately over one inch or, say, 25 mm then they will be accurate over the travel of the table (although this does not rule out the possibility of some other minor fault causing you to loose steps)