I had a similar situation converting from Nach 3 to Mach 4, USB smooth stepper to Ethernet. Below is what Andrew from the Warp 9 forum helped me with. My scales are working but I still need to add code to zero them manually.
You mapped them correctly to the ESS pins, but I would change the alias names to show the port/pin numbers, and I would also have them count 0, 1, 2 instead of 1, 2, 3 since Mach4 has you X motor start with 0 instead of 1. This will make it MUCH less confusing in the long term.
For the input signals, again, make it go from 0, 1, 2 instead of 1, 2,3
In the Mach config -> Aux positions, assign ESS encoders 0, 1, 2 and the count values. You can grab on the column line and slide it over to make the column wider.
Now everything is configured, and you need to modify your screen set. Do you want the A, B, C DROs below to show the encoder values? I would think that would be the easiest. If so, I would change the button labels on the left to show X Enc, Y Enc, Z Enc, and then set those three DROs to Motor 0 Aux, Motor 1 Aux, Motor 2 Aux....