Should I let him be?

Good day house, please i need you all to help me with constructive advice. I dated my guy for 6 years, terminated 2 pregnancies for him and now we have been married for 4 years without kids. He is now blaming me for our childlessness to the point that he even told his parents that me I aborted when we were in school. Now his mother looks at me like a demon because she is a deaconess in church. She has even told her son to remarry as soon as he can because she doesn't feel I can bear children and her son still came to tell me what his mother said, almost like there is no regard for me in their house any more. I am really tired of all the humiliation. I need you all to advise me. What should I do? Should I keep living with them or should I just let them be. Please I need hep