S[indle control depends entirely on what you want. By the sounds of it the spindle is controlled by a VFD so you could find out what input it takes to control the speed and get one of the boards that will output that signal when Mach tells it to. ie it may need 0-10v input to the VFD to vary the speed, some are 0-5v etc.
However you may not even wish to control the speed from Mach, nothing to stop you keeping it the way it is now and just manually adjusting the speed.
Motor size I am not sure on, depends on the forces required to move the axis, would think 916oz/in would be plenty especially if you gear them 2:1.
When looking for encoders make sure they are ttl.
Hood