Machsupport Forum

Third party software and hardware support forums. => Galil => Topic started by: capsteve on January 11, 2011, 10:36:54 PM

Title: unexplained stop during program
Post by: capsteve on January 11, 2011, 10:36:54 PM
ok i have everything running pretty good except the machine will just hang .....randomly during a program.
i have added some debounce and no improvement.
i do not see any triggers from limits occuring, or any errors that are present
the drivers are all still energized when it stops
the spindle continues to turn, its the motion that stops at random line numbers during the program

can the galil card stop motion because of an error. if so can i poll the gaglilcard to find stop codes or history??

1 in 10 times the program will run completely with all the same settings

i have turned all the brains off and tried, i have no custom macros

the mach history is empty

im open to any ideas, ....please...

Title: Re: unexplained stop during program
Post by: smurph on January 11, 2011, 10:45:37 PM
Steve,

Enable debug output in the plugin configuration and restart Mach.  Then try running your program.  If it fails, exit Mach and send me the GalilDebug.txt file in your plugins directory.  You can attach it to a message in this forum as well.

We will then be able to see what is happening on the Galil (communications, etc...)

Steve
Title: Re: unexplained stop during program
Post by: capsteve on January 11, 2011, 11:04:17 PM
steve,
   will do , thanks
Title: Re: unexplained stop during program
Post by: capsteve on January 13, 2011, 01:49:37 AM
steve,
  there is 5 or 6 grabfiles in the zip all were recorded after mach3 hung up....
hopefully you see some thing its all greek to me...;)
it actually took a while to get it to act up, once it did, it became frequent.

steve
Title: Re: unexplained stop during program
Post by: smurph on January 13, 2011, 12:39:01 PM
Steve,

What kind of machine are you running?  (Processor size, memory, operating system, etc...)  I don't believe this is a Galil issue.  The movement queue is drained meaning no more movement information from Mach.  So it looks like something is interfering with Mach at this point.  I'm not ruling the plugin out, but if you could give me the lay of your machine, I'll dig some more.

Steve
Title: Re: unexplained stop during program
Post by: Mogal on January 13, 2011, 01:12:29 PM
I have a simular issue. Although, doesn't seem to be as sever as Steve's.
Its completely randum, and when the same code is run a second time, it runs without error.
It can happen happened 5 minutes into the code, or 5 hours into the code.
I think its a windows issue(win 7 32bit)
I have to close Mach3, and restart it when it happens.
I set Mach3's priority one step higher, and (knock on wood) haven't seen a reocurance yet.
Title: Re: unexplained stop during program
Post by: capsteve on January 13, 2011, 01:18:02 PM
pentium 4 pc 3ghz
1gig ram
xp pro
nvidia gforce mx 440 video
contec pio16(pci)
galil 1842 motion controller
power scheme is set to always on
no screen saver
no anti virus
most non used features of windows turned off
no remote assistance
no remote desktop
no autoupdate

system restore point on drive c IS on

paging file is 1536 mb

hid stuff usb keyboard, usb trackball mouse,




pc running xp, everything non essential is turned off.
Title: Re: unexplained stop during program
Post by: capsteve on January 13, 2011, 01:24:41 PM
when mine locks up it acts like an unregistered version of mach3, it gets to a line then just sits.....but it does go past 100 lines....
when it does lock up i hit the stop (alt-a) then hit spindle to restart spindle then cycle start and it will take off where it was at..
steve
Title: Re: unexplained stop during program
Post by: capsteve on January 13, 2011, 01:29:36 PM
i did get a 40 minute cycle to complete yesterday......followed by a 2.5 minute lockup..
steve
Title: Re: unexplained stop during program
Post by: Mogal on January 13, 2011, 01:56:45 PM
when it does lock up i hit the stop (alt-a) then hit spindle to restart spindle then cycle start and it will take off where it was at..
steve

Thanks, I will try that if it ever happens again.
Chris
Title: Re: unexplained stop during program
Post by: smurph on January 13, 2011, 02:52:04 PM
Steve,

Your machine looks fine.  Heh...  with the exception of the trackball.  :)  I just can't get used to those things!

Steve and Chris,

I'll get you both a new plugin this evening.  I have Steve's email.  So Chris, send me a PM with your email address.

This new plugin is working toward release 4.5.  It's pretty different as far as how it works internally and there may be bugs in it (MPG and homing code related), so I would not suggest running it full time.  But I would like to see if any of you can duplicate the lockup with this new plugin.  If you can duplicate the lockup, then it's a high probability that it's not the plugin and we need to start looking for things that might make Mach hiccup. 

What versions of Mach are you guys running?
Title: Re: unexplained stop during program
Post by: capsteve on January 13, 2011, 03:30:36 PM
mine says R3.042.04
steve
Title: Re: unexplained stop during program
Post by: Mogal on January 13, 2011, 06:36:38 PM
Same version R3.042.40
Title: Re: unexplained stop during program
Post by: capsteve on January 30, 2011, 11:26:41 AM
Steve,
   after trying the new plugin without success, i looked over the machine. i found the encoder position wires were shielded but not
grounded. i grounded them and tested further. i thought i had the problem beat , because the machine ran flawlessly for almost 2 hours running a 20 min program.
but sadly, the machine began to stop during the cycle randomly again. i called galil to get some info on how to check for errors logged in the control card.
Tc1,sc and a few others were mentioned, after looking  i dont see any reason why it should stop. i noticed that the wdsk program used to be able to run along side the mach software, now mach has
to be shut down before wsdk can see the control card.
  to keep it simple i used the wizzard to write a small keyway slot program, straight with no arcs. my machine even locked up during a liesurely 3ipm g1.
most frequently the machine just stops with the spindle still turning, less often the machine estops in the middle of a move, a very few times it stopped on a limit trigger(not sure which one)
i think im going to try another computer as i wanter to use a smaller form factor anyway.
is there anything i need to do to windows 7, other than the usual turning stuff off??

thanks steve
Title: Re: unexplained stop during program
Post by: smurph on January 30, 2011, 06:23:30 PM
Steve,

I sent you a new plugin to try.

Steve
Title: Re: unexplained stop during program
Post by: battwell on March 18, 2011, 06:23:53 PM
any progress with this? we have one in uk doing exactly the same... pulling hair out trying to find it!
Title: Re: unexplained stop during program
Post by: steve_p on March 18, 2011, 06:28:29 PM
Hi, I'm having exactly the same problems on my router, I've got a 2160 with extended i/o running off an intel XP pro box with little else but Mach 3.042.40 installed.

It hangs in the middle of a program, spindle running, as though it's a feedhold, all servo drives enabled, sometimes Mach estops, sometimes just seems to hang, always needs a restart (otherwise next program run fails to start spindle).  It can run a file for several hours before doing this, or sometimes 10 mins.

Get "Galil lost connection" message usually. As mentioned by capsteve earlier (too many steves in this thread already!), thought all our previous comms dropout problems had been sorted when we found a broken screen in an encoder cable (noise, apparently), but still happening, albeit much less frequently. Going to reinstall on a fresh PC tomorrow.

Will try the 'start spindle, cylce start' workaround tomorrow too. Had the odd little hiccup which might shed light on it, such as today when homing, one axis just ran over both the home and limit switches, then when almost beyond the back end of the switch ramps, suddenly Mach lit both LEDs, as though it had dozed off for a second..

Battwell has done all my Mach setup, so I'm sure he'll be asking much more in-depth questions, but I just saw this thread tonight after a frustrating day and thought I'd pitch in!
Title: Re: unexplained stop during program
Post by: steve_p on March 18, 2011, 06:29:55 PM
any progress with this? we have one in uk doing exactly the same... pulling hair out trying to find it!

oh hello, cross-posted you!
Title: Re: unexplained stop during program
Post by: battwell on March 18, 2011, 06:48:59 PM
my machine doesnt seem to do this! im using a galil 2280 which is 10-100 network if that maight make a difference. (it might do it if i run a long enough file but i had it running a 3d file yesterday for an hour which it completed perfectly)

steves machine is on the same version of mach as mine .040
steves machine does act differently to mine on a few points, the main one being it requires sleep command after everything in vb scripts or it can miss doing them! mine didnt appear to be that fussy.
also of note on steves machine is explained below.
its a cnc router , ac servo driven with encoder pulses passed through amps to the galil. the machine is capable of 65 metre rapiding speeds no problem.
the test files have been cutting 3d reliefs of differing sizes, differing resolutions etc.
of note was the file seems to run fine until it stops! after this it can be restarted only by resetting mach. however unless the galil is hard reset the next file then doesnt run straight. seems to add "steps " at the end of each pass which causes the once square relief to start running at an angle of approx 20 degrees.

Title: Re: unexplained stop during program
Post by: steve_p on April 05, 2011, 03:46:08 AM
Any ideas on this random stopping problem yet? I was having several problems with my machine, so I tried replacing the 2160 with a 1750 I had on the shelf, and all my problems disappeared, apart from this one.

I previously wondered if it was an Ethernet issue, but it does the same thing with a bus controller. The 2160 was running off a custom
-made XP pc, non-onboard video gigabyte mobo, intel CPU,  nothing installed except Mach and galil stuff. The 1750 is in a brand new Advantech industrial pc chassis, intel,  4gb ram, again nothing except XP, Mach and smarterm installed. In both cases everything turned off (I think)- auto updates, power management, screen saver etc.

Several people seem to be experiencing it, any thoughts from Kenny/Smurph? Did Smurph's new plugin help? ( could I get a copy to try too, I need to run a 130-hour program this week!) thanks.
Title: Re: unexplained stop during program
Post by: steve_p on April 05, 2011, 06:39:43 AM
I left the machine running last night, doing an air cut of a 35Mb relief, and it stopped after 47,000 lines... it had done 100,000 lines yesterday on a different test before I stopped it myself.

Mach hadn't crashed, but the Galil red LED was lit. Although all the screen buttons seemed to work as they should (ie they blinked as they were clicked), there was no movement because of the Galil fault. The program couldnt be restarted for the same reason.

So I've just tried again, with debug turned on, to see if it did it again.. and it did on the first move!

Here's the debug file:

10:22:42:953  |***** DMCDiagnosticsOn *****|
10:22:50:906  Entering DMCGetHandle.
10:22:50:906  Leaving DMCGetHandle.
10:22:50:906  Entering DMCVersion.
10:22:50:906  Entering DMCmCommand.
10:22:50:906  DMCmCommand: about to wait for g_hDMCCommandMutex.
10:22:50:906  Entering DMCmCommand.
10:22:50:906  DMCmCommand: about to wait for g_hDMCCommandMutex.
10:22:50:906     Could not parse version information <DMC1750 Rev 1.2n>.
10:22:50:906  Leaving DMCVersion.
10:22:50:906  Entering DMCCommand.
10:22:50:906     Command <MG_XQ-1>.
10:22:50:906  Entering DMCmCommand.
10:22:50:906  DMCmCommand: about to wait for g_hDMCCommandMutex.
10:22:50:921     Response < -2.0000 :>.
10:22:50:921  Leaving DMCCommand.
10:23:18:328  Enmtering DMCmSetTimeout.
10:23:18:328  Leaving DMCmSetTimeout.
10:23:18:328  Entering DMCClear.
10:23:18:328  Entering DMCmClear.
10:23:18:328  Entering DMCmCommand.
10:23:18:328  DMCmCommand: about to wait for g_hDMCCommandMutex.
10:23:18:328     Interrupt Routine: received an interrupt.
10:23:18:328     Interrupt Routine: interrupt value <0>.
10:23:18:328     Interrupt Routine: controller index <0>.
10:23:18:328     Interrupt Routine: waiting for interrupt.
10:23:18:328  Entering ReadData.
10:23:18:328  ReadData: about to wait for g_hDMCCommandMutex.
10:23:18:328     readGalilString: Bytes read <0>.
10:23:18:328     readGalilString: Raw bytes <>.
10:23:18:328     Bytes read <0>.
10:23:18:328     Raw bytes <>.
10:23:18:328  Leaving ReadData.
10:23:18:328  Leaving DMCmClear.
10:23:18:328  Leaving DMCClear.
10:23:18:328  Enmtering DMCmSetTimeout.
10:23:18:328  Leaving DMCmSetTimeout.
10:23:23:265  Enmtering DMCmSetTimeout.
10:23:23:265  Leaving DMCmSetTimeout.
10:23:23:265  Entering DMCReset.
10:23:23:265  Entering DMCmCommand.
10:23:23:265  DMCmCommand: about to wait for g_hDMCCommandMutex.
10:23:23:406  Entering DMCmCommand.
10:23:23:406  DMCmCommand: about to wait for g_hDMCCommandMutex.
10:23:23:406  Entering DMCmCommand.
10:23:23:406  DMCmCommand: about to wait for g_hDMCCommandMutex.
10:23:23:421  Entering DMCmCommand.
10:23:23:421  DMCmCommand: about to wait for g_hDMCCommandMutex.
10:23:23:421  Leaving DMCReset.
10:23:23:421  Enmtering DMCmSetTimeout.
10:23:23:421  Leaving DMCmSetTimeout.
10:23:28:406  Entering DMCClose.
10:23:28:406    DMCClose: about to wait for g_hDMCOpenMutex.
10:23:28:406     readGalilString: Bytes read <0>.
10:23:28:406     readGalilString: Raw bytes <>.
10:23:28:406     readGalilString: Bytes read <0>.
10:23:28:406     readGalilString: Raw bytes <>.
10:23:28:421     readGalilString: Bytes read <1>.
10:23:28:421     readGalilString: Raw bytes <:>.
10:23:28:437  Entering DMCmClose.
10:23:28:437  Entering DMCmCommand.
10:23:28:437  DMCmCommand: about to wait for g_hDMCCommandMutex.
10:23:28:437  Entering Remove Interrupt Handlers.
10:23:28:437    Removing interrupt handler for controller <0>.
10:23:28:437      UnsolicitedMsgEventHandler: Posting WM_DMCTHREADCLOSING to window.
10:23:28:437  Leaving Interrupt Handler.
10:23:28:640  Thread shutdown complete.
10:23:28:843    Removing Unsolicited Msg handler for controller <0>.
10:23:28:843      UnsolicitedMsgEventHandler: Posting WM_DMCTHREADCLOSING to window.
10:23:29:062  Leaving UnsolicitedMsgEventHandler.
10:23:29:062  Thread shutdown complete.
10:23:29:265  Leaving Remove Interrupt Handler.
10:23:29:265       Disabling communication interrupts: calling IOCTL_GALIL_ENABLECMDDONEINTERRUPT.
10:23:29:281     Closing driver <320>.
10:23:29:296     Deleted index <0>.
10:23:29:296  Leaving DMCmClose.
10:23:29:296  Leaving DMCClose.
10:23:29:328  |***** DMCDiagnosticsOff *****|
Title: Re: unexplained stop during program
Post by: capsteve on April 06, 2011, 01:25:00 AM
im still having the same stoppage issues as well. i have replaced the computer with a with a new one containing a different brand processor and new ram. this time i used vista stripped to the bone.
i've tried setting the software priority higher in the op system as someone suggested. but it remains operationally the same as it has been, locking up randomly. the debug routine that runs in the background as a diagnostic tool always seem to show my problem occurring whenever a very specific memory address is used. don't know if its in use from other plug-ins (galil, and contec) or the op system.
steve
Title: Re: unexplained stop during program
Post by: capsteve on April 06, 2011, 01:30:54 AM
come to think of it, one of the uncertain areas i have.. does anybody know how the bios should be set as far as power management control? i get two choices neither which is off.
im thinking if the pci bus slows down or powers off their could be trouble. but i havnt seen any discussion about it so it may not matter.
steve
Title: Re: unexplained stop during program
Post by: steve_p on April 07, 2011, 09:44:02 AM
Since my last post I've been able to run the machine for several hours without a stop- over 7 hours in one case.

However, last night it stopped after about half an hour.. ???

Here's the debug file from the mach folder, there are hundreds of lines where it seems to be doing g code ok, and this section is the last bit where it seems to have reset. I'm not sure whether the section beginning with the QZ command was added immediately, or during shutdown, or whatever- basically, I'd left the machine running code and gone home, but a colleague noticed that it had stopped, so he pressed reset in mach, closed mach, shut down windows and the pc, then turned off power to the machine, if that helps explain anything..

I'd be really interested to hear if any of the guys who worked on the plugin have any ideas or can suggest things to try..?


GCode ==>  Y443.05 Z8.84

LI 0,78,-53,0<5892                                           Slots=477  Response=":"  <MoveRelative()>
LI 0,84,-67,0<6714                                           Slots=474  Response=":"  <MoveRelative()>
LI 0,95,-76,0<7602                                           Slots=474  Response=":"  <MoveRelative()>
LI 0,104,-82,0<8276                                          Slots=474  Response=":"  <MoveRelative()>
LI 0,107,-85,0<8540                                          Slots=474  Response=":"  <MoveRelative()>
LI 0,107,-85,0<8540                                          Slots=474  Response=":"  <MoveRelative()>
                    Do4MsGCode(): M=16, T=0.3280
LI 0,107,-78,0<8274                                          Slots=485  Response=":"  <MoveRelative()>
LI 0,108,-68,0<7976                                          Slots=485  Response=":"  <MoveRelative()>
LI 0,108,-58,0<7660                                          Slots=485  Response=":"  <MoveRelative()>
LI 0,109,-48,0<7442                                          Slots=485  Response=":"  <MoveRelative()>
LI 0,109,-37,0<7194                                          Slots=485  Response=":"  <MoveRelative()>
LI 0,109,-27,0<7018                                          Slots=485  Response=":"  <MoveRelative()>
LI 0,109,-16,0<6884                                          Slots=485  Response=":"  <MoveRelative()>
LI 0,109,-7,0<6826                                           Slots=485  Response=":"  <MoveRelative()>
LI 0,109,3,0<6814                                            Slots=485  Response=":"  <MoveRelative()>
LI 0,110,15,0<6938                                           Slots=485  Response=":"  <MoveRelative()>

GCode ==>  Y439.93 Z8.68

LI 0,108,21,0<6876                                           Slots=479  Response=":"  <MoveRelative()>
LI 0,108,36,0<7114                                           Slots=479  Response=":"  <MoveRelative()>
LI 0,107,68,0<7922                                           Slots=479  Response=":"  <MoveRelative()>
LI 0,102,106,0<9194                                          Slots=479  Response=":"  <MoveRelative()>
LI 0,98,145,0<10938                                          Slots=479  Response=":"  <MoveRelative()>
LI 0,94,183,0<12858                                          Slots=479  Response=":"  <MoveRelative()>
                    Do4MsGCode(): M=16, T=0.1090
LI 0,90,222,0<14970                                          Slots=506  Response=":"  <MoveRelative()>
LI 0,86,260,0<17114                                          Slots=506  Response=":"  <MoveRelative()>
LI 0,80,293,0<18982                                          Slots=506  Response=":"  <MoveRelative()>
LI 0,75,319,0<20480                                          Slots=506  Response=":"  <MoveRelative()>
LI 0,68,342,0<21792                                          Slots=506  Response=":"  <MoveRelative()>

GCode ==>  Y436.11 Z9.39

LI 0,60,323,0<20532                                          Slots=505  Response=":"  <MoveRelative()>

GCode ==>  Y438.54 Z6.44

LI 0,58,259,0<16588                                          Slots=496  Response=":"  <MoveRelative()>
LI 0,59,191,0<12494                                          Slots=496  Response=":"  <MoveRelative()>
LI 0,60,123,0<8552                                           Slots=496  Response=":"  <MoveRelative()>
LI 0,61,55,0<5132                                            Slots=496  Response=":"  <MoveRelative()>
LI 0,62,-12,0<3946                                           Slots=496  Response=":"  <MoveRelative()>
LI 0,64,-81,0<6452                                           Slots=496  Response=":"  <MoveRelative()>
LI 0,64,-149,0<10134                                         Slots=496  Response=":"  <MoveRelative()>
LI 0,66,-217,0<14174                                         Slots=496  Response=":"  <MoveRelative()>
LI 0,67,-285,0<18298                                         Slots=496  Response=":"  <MoveRelative()>

GCode ==>  Y436.11 Z9.39

LI 0,68,-330,0<21058                                         Slots=483  Response=":"  <MoveRelative()>
                    Do4MsGCode(): M=16, T=0.0940
LI 0,69,-337,0<21498                                         Slots=483  Response=":"  <MoveRelative()>
LI 0,70,-338,0<21572                                         Slots=483  Response=":"  <MoveRelative()>
LI 0,69,-337,0<21498                                         Slots=483  Response=":"  <MoveRelative()>
LI 0,73,-325,0<20818                                         Slots=483  Response=":"  <MoveRelative()>
LI 0,79,-295,0<19086                                         Slots=483  Response=":"  <MoveRelative()>
LI 0,84,-259,0<17016                                         Slots=483  Response=":"  <MoveRelative()>
LI 0,90,-222,0<14970                                         Slots=483  Response=":"  <MoveRelative()>
LI 0,95,-187,0<13108                                         Slots=483  Response=":"  <MoveRelative()>
LI 0,100,-150,0<11266                                        Slots=483  Response=":"  <MoveRelative()>
LI 0,105,-115,0<9732                                         Slots=483  Response=":"  <MoveRelative()>
LI 0,107,-79,0<8312                                          Slots=483  Response=":"  <MoveRelative()>
LI 0,109,-44,0<7346                                          Slots=483  Response=":"  <MoveRelative()>
LI 0,109,-10,0<6840                                          Slots=483  Response=":"  <MoveRelative()>

GCode ==>  Y432.64 Z9.12

LI 0,108,22,0<6888                                           Slots=482  Response=":"  <MoveRelative()>
LI 0,109,34,0<7136                                           Slots=482  Response=":"  <MoveRelative()>
LI 0,109,28,0<7032                                           Slots=482  Response=":"  <MoveRelative()>
QZ                                                           Slots=479  Response=" 5, 24, 16, 28:"  <ResetControl()>
MG_BAA                                                       Slots=479  Response=" 0.0000:"  <ResetControl()>
MG_BAB                                                       Slots=479  Response=" 0.0000:"  <ResetControl()>
MG_BAC                                                       Slots=479  Response=" 0.0000:"  <ResetControl()>
MG_BAD                                                       Slots=479  Response=" 0.0000:"  <ResetControl()>
MG_BAE                                                       Slots=479  Response=" 0.0000:"  <ResetControl()>
SP 3000000,3000000,3000000,3000000                           Slots=479  Response=":"  <ResetControl()>
FL 2147483647,2147483647,2147483647,2147483647               Slots=479  Response=":"  <DisableSoftLimits() State.nAixs = 5>
BL -2147483648,-2147483648,-2147483648,-2147483648           Slots=479  Response=":"  <DisableSoftLimits() State.nAixs = 5>
GM 0,0,0,0                                                   Slots=479  Response=":"  <SlaveALL(false)>
GR 0,0,0,0                                                   Slots=479  Response=":"  <SlaveALL(false)>
DP 81813,256449,-26492,99061                                 Slots=479  Response=":"  <ResetControl()>
MT -1.0,-1.0,-1.0,-1.0,-1.0                                  Slots=479  Response=":"  <ResetControl()>
AB 1                                                         Slots=479  Response=":"  <ResetControl()>
MO ABCD                                                      Slots=479  Response=":"  <ResetControl()>
CN 1                                                         Slots=479  Response=":"  <ResetControl() Limits low checked!>
CN ,1                                                        Slots=479  Response=":"  <ResetControl() Homes low checked!>
KPE=0                                                        Slots=479  Response=":"  <ResetControl()>
KIE=0                                                        Slots=479  Response=":"  <ResetControl()>
KDE=0                                                        Slots=479  Response=":"  <ResetControl()>
TM 1000                                                      Slots=479  Response=":"  <ResetControl()!>
OP 0                                                         Slots=479  Response=":"  <UpdateOutputs()>
Title: Re: unexplained stop during program
Post by: steve_p on April 10, 2011, 09:53:05 AM
my machine just did the unexplained stop after running a 3D file for 40 hours (it was only half way through though..!)

No Galil error LED, elapsed time still running in mach, but g code window halted.

I've noticed that after this happens and I hit stop (tried feedhold then start- doesn't do anything), mach won't let me jog even though I can click the jog button to make it green. I have to do a reset of mach before I can jog again. It seems like the user interface of mach is fine, but it's not talking to the galil any longer, although there doesn't seem to be any fault in the galil..?