No, it is never too late to seek God's forgiveness, because He wants to forgive us and make us part of His family–both now and forever.
The only person God cannot forgive is the person who never seeks His forgiveness. Sometimes people refuse to turn to Him because they don't believe in Him, or they want to keep on in their sins. Sometimes they refuse because (like you) they simply can't believe God would forgive them. But whatever the reason, God loves us in spite of our sin, and He yearns for us to turn to Him in repentance and faith so He can forgive us.
How do I know this? I know it for one reason: God sent His only Son into the world to make our forgiveness possible. God is righteous and just, and He can't simply pretend that our sins never existed. Sin must be punished; otherwise God is not just. But listen: Jesus Christ came to take upon Himself the punishment we deserve for our sins. He was without sin–but on the cross, all our sins were laid on Him, and He died in our place.
Think of it: All your sins–including this one that has haunted you for so long–were placed on Christ, and He took upon Himself the judgment you deserve. Confess your sins to Christ and trust Him to forgive you–and He will. The Bible says, "If we confess our sins, he is faithful and just and will forgive us our sins and purify us from all unrighteousness" (1 John 1:9).