Sounds like you have an X axis home switch enabled, even though you don't have a switch. If no switches are enabled (pins assigned), then it shouldn't move.
You can use Ref All without switches, and it should zero all the axis at there current position. Before I had home switches, I would jog the axis against hard stops and use Ref All. This gave me a repeatable home position, just like having switches.