This phrase means that the speaker will always remain loyal to the person they are talking to, even if that person often leaves them or “abandons” them emotionally. It implies that no matter how often the other person leaves them feeling hurt, they will not give up or let go of the relationship.