Hood,
No I'm not sure the gibs are setup properly. I recently picked up this mill from someone else and have been learning to machine on it (I'm not even familiar with non-CNC machining) so I've not tweaked the mill yet. I need to adjust out the backlash too.
But I ran a number of tests today. I do get that same problem (periodic "waves") with a straight diagonal line though. This is 0.375" thick...

I then adjusted out (in software) the notch/divot at the cardinal point due to the backlash compensation stepper reversal, and reducing the backlash speed from 50% of max to 10% seems to have worked beautifully.
I also played with different feeds around an arc and the repetitive waves stays mostly the same, though a little less pronounced at lower feeds. I also noticed that the finish is much better at lower feeds. Here's the best I got at the end of the day, which is a 0.010" finish pass at 0.5ipm and 1500 rpm. The material is still 6061-T6, approx 2" wide x 0.375" thick. This photo makes it look a bit better than it actually is, but still it's much better than yesterday.


Still a lot of tuning and tweaking to go, but on the mechanicals of the machine. We're leaving the scope of this forum now, so I'll go research that quietly

Cheers,
-Neil.