I know you feel like you're being torn in two–and in a sense you are, for you are facing a very difficult situation. I pray, however, that you will stay faithful to Christ, and trust Him for the future.
Think of it this way: If you could look back over your life a hundred years from now, what would you conclude was the most important decision you ever made? It would be your decision for Christ–because it determined your eternal salvation. To put it another way, a hundred years from now, you wouldn't trade anything for that decision.
More than that, in heaven you will realize just how much God loves you, and you will realize just how much Christ suffered for you. The Bible says, "He was despised and rejected by men, a man of sorrows, and familiar with suffering" (Isaiah 53:3). If He suffered this much for us, shouldn't we be willing to suffer for Him?
You face a situation that isn't common today–but it was to the first Christians. To them the Apostle Paul wrote, "If a woman has a husband who is not a believer and is willing to live with her, she must not divorce him. But if the unbeliever leaves, let him do so" (1 Corinthians 7:13, 15). I pray it will not reach this extreme in your case, but that your husband will realize his own need of forgiveness and yield his life to Christ.