At 3000rpm and a 10mm pitch and 2:1 reduction then you would get a max of 15,000mm/min.
That is of course if the motor can spin at 3,000, if it is a DC servo then it may not reach that rpm, it will depend on whether your power supply to the drives is full motor voltage , usually it is a bit less.
I would personally set the Velocity in Mach at 10,000mm/min and see how it goes.
Not sure what you mean by pulse per minute for the encoder, presume you mean pulses per rev?
I will also assume that it is actually the line count as 500 was a common line count of an encoder.
So for Steps per unit it would be
500 x 4 for encoder x 2 for reduction then divide by 10 for pitch = 400 steps per mm.
Hood