Machsupport Forum

Mach Discussion => General Mach Discussion => Topic started by: rprichard on January 09, 2015, 02:20:40 PM

Title: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: rprichard on January 09, 2015, 02:20:40 PM
OK...So I just spent about three hours debugging a 250 line G-Code file that Mach 3 was not executing correctly:

Script description:

A simple wood box cutout with an internal pocket subroutine, external frame cut-out subroutine, and two subroutines to handle two sets of drill holes..
This code was working until I modified it last night to add some new routines and make the code more structured and modular (as much as it could be with G-Code as the source language).. The file was resident on my Windows-7 computer that I use for CNC milling the file system was  and mounted on my McBook Pro.. After three hours of diagnosing the G-Code, I was seeing some additional blank lines that were not showing up in Notepad or Textedit on my Mac.. I loaded the files into a Hex Editor (and after spending about five minutes) saw several non-on printable characters that were out of the Hex Range of the printable (displayable) character set that you could see in any of the ascii editors like Notepad on Windows.

These individual records from the file were being loaded by mach 3 with no error messages or a message that trying to exit a subroutine that but there hadn't been a subroutine call.

If I didn't have over 40 years of Software Engineering experience, I would still be sitting in front of my computer, deleting, adding, modifying lines of the file  trying to figure out what to do..

So here is the problem that Mach 3 apparently has:  When it opens a file, and loads in each record, it attempts to parse each line without first checking if there are invalid characters contained within.. There are built-in C ascii functions & C++ methods functions for checking character validity.

Solution:
Mach 3 should first parse each line in the G-Code and notify the user that non-ascii/invalid characters have been found ajt "line number" of the input file and offer 1 of two options...1) cancel file load.. or 2) continue replacing invalid characters with valid "white space" "\040"...

I'm hoping this is such a fundamental error checking step, it has found it's way into MACH 4 and the developers have not just copied and pasted the existing file loading and parsing routines..

Reuben M. Prichard Jr.
Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: mrprecise44 on January 09, 2015, 03:20:23 PM
Hi rprichard:
Mach3 was not designed to de-bug files inadvertently written with non-printing characters. You  could take a perfectly good g-code file written with a text editor like "notepad", or produced by one  of the many wizards that performed a simple operation, and re-write it with MS Word you use for letters etc.,  and re-load the "Word" created version in Mach3, and it will not run.
This is a pretty basic requirement for CNC machine g-code files. You might find one of the books on CNC programming by Peter Smids helpful, as he explains every aspect of CNC programming in detail.

John
 
Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: rprichard on January 09, 2015, 03:41:30 PM
John,

Thanks for the quick response...

Performing validation of all inputs to a program...both inter- and intra- field is a requirement for all software development applications: data must be validated for set-inclusion, set-exclusion, range, type and valid relationship with other fields and parameters.. This is FUNDAMENTAL! Failure to perform adequate data validation cost both the developer and the target end user both time and money..

I have a BS & MS in computer science and worked 31 years for AT&T Bell Laboratories and have worked with many of the people that brought you fiber optics, the Unix operating system, and other telecommunication devices. I also have a Motorola Digital Six Sigma Green Belt in process/quality improvement.

Not implementing fundamental data validation is typical of an immature development organization that:

1. Has inadequate functional requirements documents
2. Does not have code reviews
3. Has inadequate testing... function, unit, and system level testing
3. Demonstrates no level of quality processes (ISO 9000)
4. or just doesn't have adequate knowledge of the software development cycle (requirements, development, test, support/maintenance)
5. Or a combination of some or  all of these.

Software needs to be modeled on "how it will be used" by the end user.. In the last 20 years, network devices, shared file system across multiple OS & hardware environments is more the NORM rather than the exception..

The fact is, the module that handles G-code source processing needs improvement..

Please help me make your product better....forward this request on to the Product Manager in charge of Mach 3 & 4...

This is not "Rocket Science"...It is fundamental.

R. M. Prichard Jr.
Colts Neck, NJ
Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: BR549 on January 09, 2015, 04:30:59 PM
Obiviously you have not been in the CNC Gcode field very long (;-)

Just a thought, (;-) TP
Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: mc on January 09, 2015, 06:02:52 PM
I think a good analogy for G-code, is to think of it like a legacy programming language from days when processing power and memory was costly, and it just wasn't viable to error check it fully, so you had to rely on making sure what you typed in was correct.
At least Mach3 pre-reads the complete file and flags up errors, unlike lots of machine controllers which would simply run then fault out at the problem line.

I'm not trying to defend Mach3 for lacking what is now considered a basic feature in nearly all programmes, however Mach3 dates from when processing power was still limited, and from an area of industry where it is expected that G-code would either be created using a basic text editor, or from an automated post-processor. Non-visible charcters simply shouldn't be inserted using either of those methods.
Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: rprichard on January 09, 2015, 09:54:46 PM
MC....I know those vintage programing languages quite well! You have to keep in mind the cost of fixing a problem in the field is at least four times as much as catching it in the development cycle.. I've been around a long time and have a pretty good idea why Mach 3 is the way it is... In 2015, there is no excuse for weak engineering practices.. I can fully understand missing complex validation rules...but not implementing this this basic level of validation is inexcusable...especially given that the validation is as simple as:

boolean no_bad_chars=true;

while ((record=getrecord)!null){
   for (i=0;i<record.lenght();i++){
   if (!isascii(record){
          record=" ";
          no_bad_chars=false;
      }
   }
}
If (!no_bad_chars){
   putout and error message
   and anything else you want
}


That's it written in meta code..

The fundamental problem is that people cut and past form multiple applications to create scripts..they don't just use notepad.. I have a excel spreadsheet that automatically calculates bit offsets metrics conversion to the point that all I have to do is "copy" and paste the columns into a text file.. That's how people work today.. This is not the 60's when G-Code was written!.....I was around then in my 20's

RP



Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: Tweakie.CNC on January 11, 2015, 06:31:28 AM
Hi Ruben,

So far I have not used any software that does not have it's own set of shortcomings. Perhaps that is the way with everything in life.
In real terms, if you want to use Mach3 / 4 then you have to be prepared to learn it's peculiarities and work within it's capabilities. All of us here probably moan about different things at one time or another, some of Mach3's problems get fixed whilst others don't, but that is just the way it is - take it or leave it.
For my part, I find (or others here on the forum, kindly, find for me) a suitable 'work around' to anything that causes me problems so it's on-wards and up-wards and never look back.  ;)

Tweakie.
Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: rprichard on January 11, 2015, 07:01:07 AM
Tweaky.... Your point is valid regarding software all having it quirks or short comings....however they are mostly operational, lack of functionally,....but Mach 3 is "fed" by not only humans but applications...because of this, it is incumbent for your application to provide as robust a loading interface as possible...this is just "common sense!"...especially given it is a couple lines code that would fix this BUG are for the most part already part of the string class library of the programming language being used to write the code.. This might be a one or three line change to implement.

This is a "BUG" by any name...It may not be a severity 1, or 2.. But just think of the ramifications... I wasted three hours of my time looking a reason why Mach 3 loaded the file, gave no notice that there was any problem, but as soon as I ran it ...it just kept skipping over all the subroutines and behaving erratically....I made the incorrect assumption, that your I/O process would inform me of any problems loading the file. It did not.

You have two options.... either write an MR(modification request) against the loader routine or do nothing. If you do nothing that says a lot about your companies focus on quality! A third option...You can also just send me the loader module source code and I'll fix it...(which I'm sure you won't even if I sign a NDA)..

What I'll do today is write a simple java application (which will be platform independent) that will simply read any file and tell you if you have any "non-white space" characters and give you the option of replacing the with \020 (" ") or spaces.. As soon as this is done, I'll post it for anyone that wants to run their g-code through it before the load it into Mach 3.

With this "deal with it" attitude that you represent....I'll tell you customers vote with their "pocket books!"

Reuben M. Prichard Jr.




Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: Tweakie.CNC on January 11, 2015, 07:31:40 AM
Hi Ruben,

I don’t disagree with anything you say but in reality this is the way Mach3 is and it is unlikely to be changed. Mach4 may be a different story but that’s something for the future.
All of us, are always free to vote with our ‘pocket book’ in fact some potential users of Mach3 choose LinuxCNC or other CAM software (which is free) the choice is ours to make but at the end of the day it all comes down to what we think works best for us.
Your approach, of using a Java ‘work around’ is, in my opinion, the way to go – it solves your problem and that is all that is important.

Tweakie.
Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: rprichard on January 11, 2015, 07:50:49 AM
Tweakie....

Thanks for the response... Could you validate that this is being handled in Mach 4 Correctly...IF not please write an MR so a fix can be scheduled..? It is my understanding that Mach 4 will be released within the next few months..

R. Prichard
Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: rprichard on January 11, 2015, 08:03:06 AM
Tweaky,,,,

Your quote ".....but at the end of the day it all comes down to what we think works best for us!"....and what about what's best for the customer? your quote shows a total disregard for your customers....Hopefully, this is not the view our your senior management who I will contact regarding this.

My name is Reuben not "Ruben/Rubin..."... In the end it's about attention to attention to detail...that makes a company a success!

Reuben M. Prichard Jr.... A person that FULLY understands software, Quality processes, and the software development cycle
Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: Tweakie.CNC on January 11, 2015, 08:07:35 AM
Hi Reuben,

Mach4 has already been released http://www.machsupport.com/software/mach4/ and some of us are using it successfully despite the understandable delay in getting all the external motion controller software’s up to full functionality. It all takes time, and it is still early days yet, but it will get there in the end.

As for your particular problem of the non-recognizable characters – perhaps if you used a CAD / CAM software which has a Mach3 post –processor you would not have to resort to ‘cleaning’ the Gcode files before passing them to Mach3. Just a thought.

Tweakie.
Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: rprichard on January 11, 2015, 08:10:32 AM
Tweakie..... Please spell my name correctly!...

Just fix the problem...and stop trying to deflect the solution to some body else...It's your problem (Artsoft) not another company.


REUBEN.
Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: ger21 on January 11, 2015, 08:28:15 AM
Tweakie doesn't work for Artsoft. (and neither do I or most other moderators here.)

Quote
But just think of the ramifications... I wasted three hours of my time

In 10 years, this is the first time this issue has ever come up, to my  knowledge.

And Mach3 is no longer being developed, and bugs are no longer being fixed. The last update was over 2 years ago.


Fwiw, there are actually far more important bugs that yours in Mach3 that will never be fixed either.

And I think that just about all CNC control software comes with a disclaimer that it's the user's responsibility to verify that the g-code they are using is correct.


Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: Tweakie.CNC on January 11, 2015, 08:30:37 AM
Sorry Reuben but this discussion is getting nowhere.
If you fail to listen to advice then I suggest you go elsewhere.

I do not represent Artsoft, I am just a moderator on their forum.

For my part I use non-standard Mcodes such as M10/M11 and M62/M63 and in all cases I have to modify the CAD / CAM post-processors that I use. It's no big deal - I don't have your years of experience, in fact I don't have your years, but post-processors are easy enough to modify if it solves the situation. Why don't you just do the same ??

Tweakie.
Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: rprichard on January 11, 2015, 08:31:31 AM
I'd love to respond but I'm busy writing an email to New Fangled Solutions in Livermore Maine..
Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: Tweakie.CNC on January 11, 2015, 08:33:16 AM
 ;D ;D ;D

Tweakie.
Title: Re: Mach 3 Appears to not handle non-printable characters in Ascii files.
Post by: RICH on January 11, 2015, 10:01:44 AM
Mr. Reuben M. Prichard Jr,

Your qualifications are impressive and talents can surely be used here. I hope you would consider contributing additional
suggestions in a positive manner. Maybe have a look at Lua Programming as that will be used for Mach4. The Mach
community of users always benefits from another users talent.

Good show on an email to Artsoft and hopefully they will add it to their list of things to do for Mach4.
That is a positive action.

By all means post any programs that would assist users.

BTW, Mach has developed over the years via concerns, desires, and suggestions provided by some 25,000 users.
There are "bugs" or limitations in the program. Tweakie's assistance has been helpfull to many users,
maybe it didn't satisfy what you wanted, no need to get on him for that.

As for spelling of your name incorrectly...lighten up my friend!  this poor Lithvac's name is crucified ever day......
and I have never experienced a Six Sigma Black Belt or Master get sniffy over spelling!
Nor my refusal to bow to them....they earned my respect and and many never got it!

Enjoy the site, contribute out of the goodness for giving to others as that is self reward you will come to cherish.

RICH