Try lowering you velocity and accel in motor tuning. Say 50% and see what happens.
RICH
Hi Rich, you know what's funny, I had seen a post in my searching that said to cut them in half, so I had done that. Didn't work. In the spirit of things, I cut those in half again and guess what, it worked. I went from the 120 and 10 that the pdf shows to 50 and 5, then to 25 and 2.5. That last one works just fine (only did X this morning, will do the others when I get home) and doesn't seem to be very slow at all.
On your video, the first run (continuous) looks and sounds like a classic resonance problem (growl) followed by a stall (higher whine). The second run (step) looks and sounds like resonance with (possible) missed steps but with no actual stall.
Two other things I noticed from your video: You have Max NC 10 Wave drive selected whereas in the PDF you linked to he's using Max CL Mode. The other is a lesser point: He has an output set to pin 10. Pin 10 is an input so this makes little sense.
In one of the first paragraphs in the pdf it mentions that post-2004 machines have the same board for open and closed loop, so the CL option should be used for both. From my understanding, the wave option is for pre-2004 boards that are open loop. I have no idea about the pin 10 setting in the pdf, will research that a little more to try and understand.
Now to work on the spindle, assuming the Y and Z mimic the X settings that are now working. I saw a few people just control the spindle with a variable switch. I might just do that since I have used the same exact thing to control a mini lathe in the past.