This song is about a man who does something to lose his woman's trust, most likely cheat on her. Anyway she say's she forgives him and they can try again but it's just not the same anymore. There's no trust there, and love is just dying and dying. He's trying to make it work but it feels like "moving mountains" So in the end he's just like lets just end this, that way you can be happy and I start over. Hence the words:
"Just leave me, be"