Your're right the swarf would need to be extracted, and a sketch would help...
I experienced a similar thing to you - after a perfect skim, - even using a square spiral to take any z axis irregularities out, the job still cut poorly because of highs and lows. The problem was the vacuum was compressing the bed down slightly. Obviously when skimming it wasn't turned on...
Lets hope we get some clues soon..
Jim