I do a lot of work day to day with VFD drives. You CAN place them in the cabinet with your controls, but you need to be very diligent about following proper grounding and EMI mitigation practices. This includes having a galvanized or SS subpanel in your enclosure (or removing the paint from under your VFD), having a master grounding bar with braided grounding straps going to each subpanel, grounding the enclosure body individually, etc. Physical separation of AC, DC and VFD output wires is a must to prevent noise from coupling onto your control wires.
Even then you do risk having issues with noise. If you do place your VFD in another box, you really should still be following all these practices. Shielded cables are a lifesaver, but can get a bit expensive.