HI Pavlic,
I think first of all you are going the wrong way with you encoder counts
You really want more counts, not less. With the DSPMC set up for 20k steps per inch, if I some how loose a step, it is .00005 of an inch. It takes a lot of lost steps to make a difference. At 800 steps per inch as you have, a lost step is ~.0015. a huge difference and a big difference in the smallest move that can be made.
You also say that you have 800 steps per inch. At what encoder setting? When you change the encoder counts/rev you have to reset the steps per inch.
Mike