Stephen,
Just a shot in the dark but if it was lost steps (caused by whatever) then the finishing pass, with the small endmill, may have more small Z axis movements and thus show more error.
If you are satisfied that the problem is not mechanical then you should perhaps start looking for electrical noise and thoroughly check all electrics etc. (I know this is easier said than done

) A search for electrical noise should bring up some suggestions.
Tweakie.
btw. Welcome to the forum.