To which input on the Gecko have you connected the output of the C3 card?  As you say, whichever one it is the corresponding input needs to be enabled as Index in Ports&Pins.

As the 540 as far as I can see doesn't have a 5V output I assume that you have a separate 5V supply for the C3.  You will need to connect the - terminal of this to the 540 ground.

The reason for the open-collector option is if the C3 is being connected to a device that has a different supply rail, such as 12V or 24V.  As (I think) the 540 is 5V as is the UC100 you should use the TTL option and do not require a pullup resistor.

The card should output either a high or low logic level while the disc slot is allowing light to pass in the opto interrupter, and the opposite when it isn't.  The output is therefore a pulse of width corresponding to the slot width at the rotational speed.  If the pulse is low in the first case then set active low in ports&pins, otherwise leave active low with a red cross.

Happy to help, I've had a lot of assistance on here over the years, just passing it on.

Just to add on my system the Index is connected to Port 1 Pin 10 and I think is active low.

Oops!  Just checked and it's active HIGH - i.e. a red cross in the active low column.  Actually I think Mach3 looks for the edgen so it shouldn't make much difference.

In P&P in the Spindle Setup tab you need to tkc for "use spindle feedback".  In General Config I have Index debounce set to 10, i.e. 400 microseconds.

And in answer to another question, yes best to update the uc100 software to latest version from the cncdrive website.  Instructions are there too.

One thing, there are apparently lots of uc100 rip-offs around and if you have one of those you're unlikely to find it works.