I have a fineline automation machine dual rack and pinion.
Does anyone know what seems to be the best accl and vel for these without losing steps?

Test to find out. You can calculate it but just as easy to test as you will still need to verify even if you calculate.
RICH

I am currently running acceleration at 25 ipm and velocity at 150 ipm.
I dont know if this is way too conservative
How do I test without a lot of trial and error time?

Assuming your using steppers. Set the velocity for say 10 ipm and accel for 2, jog the machine back and forth
over a distance, if no skipping, then keep doubling  the settings until it skips. At that point reduce from the max 1/2 the doubled setting when it skipped ie; 150 -300, then use 225. That should be rather close to max velocity. Test / play around  that setting to confirm.Now start increasing the acceleration until it skips and when it skips reduce that to 1/2 the doubled setting when it skipped. Now you have a rough max velocity and acceleration. Now reduce velocity and acceleration by 30-40% to leave some headroom for machining. You can refine that setting over over time.

You need to try some cutting of different materials using different feedrates and depth of cuts. Then you'll have some experience on actual machining of a piece.

Dosen't take long to do........,
RICH

Let me make a general comment. You should be  interested in realiable motor power. That accurs where torque x speed is max and falls in a rather small range  on the motor curve. You need to decide what is more important ie; velocity, accel or power. They are all interelated.
RICH