If it is only when the VFD is running then it is very likely that the issue is noise. Debounce in Mach will not make a difference if it is losing steps as it would be external wiring to the stepper drives that is seeing the noise. You need to shield and properly ground all your wiring, especially any low voltage signalling wiring.
I use 24v on all my machines where possible to help eliminate the effects of noise and for signals that have to be 5v such as encoders or Step/Dir signals I have them via line drivers to make them differential (Step +, Step - , Dir + Dir -) This makes them very noise immune. My servo drives accept differential step/dir so I dont have to convert it back to single ended for them but if you went down that route then you could easily convert back just before the drives with a line receiver.
As said though properly grounding and shielding all your wiring will help greatly with noise issue.
Hood