Doc,
For starters, serial is woefully inadequate for this project. Get an ethernet crossover cable and configure the system to use the ethernet communication. When you have done that, if you are still having trouble, please send my a copy of your .XML file from Mach3 so that I can look further into this. There is no way that the serial port will give anywhere near acceptable performance for Mach.
Kenny