Hello everyone!

I am attempting a project where I have my 5axismaker cnc machine have computer vision capability. The major idea is that the object that my cnc is operating is freestanding and as such, is subject to occasionally move. Since restricting the movement of the object is not an option, I want to instead add cameras to track the object and then communicate with the cnc to adjust accordingly so that the predetermined toolpath remains consistent. As a novice in cnc operation, I currently input my g-code into mach4 and let the process run but could someone here provide me with some starting points for how I would have it so that the inputted g-code in mach4 can be constantly updated based on the position of the object? Thanks!!

My immediate thoughts is that computer vision is unlikely to have the precision needed to measure the movement to the accuracy required.  Why can't you fix the workpiece?  Also I don't think that constantly changing the g-code would be a good approach - for one thing once it is read into Mach it can't be edited without stopping execution.  More likely might be to change the offsets to account for a move, but what if it rotates?  Unless the material is very soft, if it isn't fixed down to the machine it will move unpredictably in response to cutting force, possibly in ways that whatever algorithm you devise has not anticipated.

Hello,

I forgot to mention that instead of removing material, my project will be depositing material similar to a 3D printer. I have gone through some other projects where they have implemented this idea as a method of correcting inaccurate deposition and I am aiming for something similar but with a cnc. I don't necessarily need extreme precision but definitely enough that I believe I can obtain with computer vision. I have read of parsing the g-code into smaller segments and then correcting future executions but I was hoping if someone could provide more details as to how I can in real-time adjust future g-code executions? Thanks!

You may be asking in the wrong place as this forum is specific to Mach 3/4 but I think your machine uses a different controller?

I do use mach4 as the controller. Currently, I input my g-code into mach4 gui and the cnc fitted with an extruder deposits the material onto the object. But it has no feedback control if the object moves. I am hoping there is a way to in real-time, update the instructions on the printer without actually stopping the printer.

The problem with this is that withing Mach4 the gcode is parsed and fed to the motion controller as a stream of instructions, and in fact the flow is 2-way as the controller interacts with Mach4, for example to tell it to stop sending while the motion buffer is executed.  That interface is proprietary so can only be accessed by companies making MCs with an appropriate agreement.  You might have better luck using something like GRBL or one of its derivatives, as this has an open and published interface.  GRBL is normally sent g-code commands from a program such as UGS (there are several others) - you would need a version that can take input from the MV system, compute any moves that have happened, calculate the necessary changes and update the remainder of the gcode.  It all sounds very hard!!  Why can't you just stick the shape down?

I do know there are some motion controllers that offer a closed loop feedback system.  Vital System's HiCON can for one. 
Having never used a 3D vision software before, I have no idea what the output data would look like, but if it is accurate enough to distinguish X, Y, and Z positons, you may be able to convert those into data that the feedback system can use.  The HiCON uses encoder feedback, whether that is from an rotary encoder or from a linear encoder, it is still A, A-, B, B- commands.  If your software could be converted to steps to feedback into this you may be able to get something to work.  Just a thought... throwing ideas out there.  Maybe one will stick.

I think that's not quite what the OP wants.  The HICON system can take feedback on position of e.g. a slide from an encoder and use this as feedback in the controlling servo, rather than relying on the screw precision.  But it is sent commands from (e.g.) Mach4 just like any other controller.  AFAICS he would like to modify the g-code in response to movements of the workpiece - so for example if some force from the extruder rotated it on the table it would in effect rotate the coordinate system to correct.

Yes exactly, as an example, adjust the coordinate system to correct in the case that there is motion. I see a lot of computer vision based projects for having a camera pan and tilt to track a human face for example which is an implementation that I want to simulate on a cnc but it seems that unlike a robot camera which can be run using pyhton, the actual interface between the cnc controller software and motion controller is more complicated. I am open to any ideas as I am still a novice and thank you!

As a proof of concept, I would also even want to start with a simpler example where I just attach a depth sensor (ultrasonic/proximity for example) to the cnc end effector and attempt to control the position of the end effector by having it maintain a consistent distance from an object. My original idea would be then be a huge step up from this example but I can at least get the idea of how to modify the cnc arm.