Ok, good.
I'd also try communicating at 19200.
What you now need to do is set up a number of modbus comms configurations.
setup a cfg say #1, to write two coils starting at address 0001h
This will allow you to start and stop the spindle in both forward and reverse
setup another cfg, say #2 to write the holding register 0002H
This will allow you to set the inverter frequency.
Once these are setup, Mach will be continuously writing these coils and holding registers to the VFD.
Now you need to set the values. For that you need to write a brain. In the brain you will take the set spindle speed DRO put it through a formula and then write the value to the Modbus cfg#2 data buffer.
The formula will scale the speed so that it will send 4000 to the VFD when your maximum spindle speed is asked for. If your max spindle speed was say, 24,000 rpm the formula would be;
V = Set Spindle speed DRO /(max spindle speed/4000)
V = Set Spindle speed DRO /6
As Mach3 is sending this buffer to the VFD, it will receive the Frequency value.
You then need to write the Spindle ON LED and the Spindle Direction LED to CFG#1
That should do it.

You should look at the video tutorials on brains and serial Plugin Modbus.
Cheers,
Peter.