Hi,
From my opinion speed of fast encoder (hardware interrupt, but not dedicated one) should be able to handle this, we can notice from the pace of counting and few error of counting.
There should be a bug need to be addressed. I know that finding this mysterious bug is not easy.
I'm sure you guys can do it, It's just a matter of time.
Thank you for your work,