Insecurities

I have realized a lot about myself recently that I am not happy with.  The question is....how do I change it?

I never really think of myself as an insecure person.  But recently, I have realized how untrue that statement is.  Looking back, I can pin point what caused it, but what I can't pin point is how to get rid of it.  Does it just take time?  Funny thing is, I had this epiphany and discovered that my insecurities have nothing to do with my weight!  WHAT you say?...  I know!   How could my insecurity not be about my weight.  No, I realized that all my insecurities are about my insides, and have nothing to do with my outsides.  OK, I can't really pinpoint when it started, I can pinpoint where it was extremely exacerbated.  It started a LONG time ago with my history of being rejected.  Starting with my mom when she was with her long term boyfriend.  (Sorry mom, not judging or condemning, it's just fact.) Then with my grandparents, who seemed to kick me to the curb for my child.  And with my child, who seemed to want my grandparents more than he wanted me.  The point where it was exacerbated was during my marriage.  It took rejection to a whole new level.  Not being touched or loved for 7 years, makes your mind do very strange things.  And you are always wondering...What is wrong with me?
The reason all of this is coming out is because in my current relationship (which is still going wonderful by the way)  We are getting to that stage where things are more comfortable, and we're not spending every waking moment with each other.  And I can predict the days where I think he is going to stay home instead of spending the night.  It's usually because he's had a bad day at work, or just because he got home from somewhere really late, and just wants to go home.  I can know these things ahead of time, but still feel completely rejected when they happen.  I can know all the reasons he has for wanting to stay home,  or know that he's had a really bad week at work and that he kind of closes down when that happens, and rationally know that they have nothing to do with me, but still I feel melancholy, and the little voice in the back of my head is saying...ok, what did you do to make him not text you, what did you do to make him not want to come over.  Is he tired of my child acting up?  Do I bother him too much with how much I have to get up at night, and does he now need a night free of me?  Is he tired of..........  Is he starting to realize..........
Am I that afraid I'm going to lose him?  Or am I trying to guess at rejection that is non-existent, so I don't get so hurt when it finally happens?  I guess it's hard for me to believe, that a normal, wonderful guy wants to stay with me.  I feel like any day now, he is going to wake up and realize he wants nothing to do with me.  Something my friend Jessie said is sticking in my head. At girls night, I had told her that he had invited me to Indonesia, when he goes diving out there in Feb or Mar.  I was letting my friends know, because I would probably need help with babysitters.  I also told her that I was preparing as if I was going, but I'd give him a way out, just in case he didn't realize how expensive it would be to take me along.  After all, he did ask me at a wine dinner.  And Jessie got very serious and looked at me, and said in a very strong voice, that I was selling myself short.  That she is sure if he invited me that he meant it, because I am worth it. 
I want to be worth it, but the problem is that deep down, I don't feel worth it.

        Question is how do I change this?  I don't want to be this way.  My rational mind is telling me that I have nothing to be insecure about.  And I need to stop awaiting doomsday.  How do I get this feeling to go away?  It would be nice just to have the normal insecurities with my body, I am used to those.  I know how to deal with those. ;)