The most important thing I can tell you is that God loves you, and He doesn't want you to carry this burden of guilt any longer.
Furthermore, He has done everything possible to lift that burden from you. Our greatest problem can be summarized in one word: sin. It isn't only your problem; it's the problem of the whole human race. Because of sin we are separated from God, and down inside we know we haven't measured up to His standards.
That's why we feel guilty—and we feel guilty because we are guilty! The Bible says, "My guilt has overwhelmed me like a burden too heavy to bear" (Psalm 38:4). But we can't get rid of this burden on our own. We can deny it or suppress it, but we can't eliminate it by ourselves. Only God can take it away.
And that is precisely what He did when He allowed His Son, Jesus Christ, to die on the cross for us. Christ was sinless and pure—but on the cross all our sins were placed on Him, and He took the punishment and death we deserve. The Bible says, "God made him who had no sin to be sin for us" (2 Corinthians 5:21).
Don't carry the burden of your sin and guilt any longer. Instead, open your heart to Christ and by faith give your burden to Him.