Scott,
Have a look at the Kress milling spindles.
They are like a Dremel on steroids, I have some of the old fixed speed ones, 28,000 rpm and to be honest they are noisy but not as noisy as a router.
Recently bought one of the new 800W models which goes from 10,000 to 30,000 in 6 thumb wheel stages and in 1,2 or 3 they are very quiet.
They come with an 8mm collet as standard so it gives an idea on the build.
They also do a 1050 W model.
John S.