The type of switch isn't as important as the quality of it. Either one are susceptible to noise which is what it sounds to me is your problem, noise. But, if I had my rathers, I would use mechanical switches as part of the e-stop circuit for the limit switches and use prox switches (24VDC) for the home switches. I am a firm believer in using 24 VDC for everything except step and direction, index signal, probes and MPGs. This requires more components, money, space, etc. You can, as many others do all the time, use 5V with great success but you have to be very mindful of noise. 24V has almost 5 times the noise immunity as 5V. Shielded cables and good wiring practices are the 2 main factors of getting the results you want, good clean consistent and accurate state changes. Good practices should be followed no matter what the voltage or what your wiring.
Brett