You didn’t have to try making your husband’s friends like you. You didn’t have to go out of your way to “make” them accept you. It’s THEIR fault they are racist, not yours. THEY should have made you feel comfortable, accepted and valued. Attempting to turn racist people into less racist is a waste of time; and more importantly, they don’t deserve your presence in their lives.
It’s a good thing your inner circle has changed over the years. There should be no place for racists to begin with.