No, they aren't too young to learn about God at all. They may not be able to learn everything about God, but they can learn something, and that should be a great encouragement to you.
Do you remember Jesus' reaction when His disciples tried to keep some children away from Him? The Bible says He was indignant, and then went out of His way to welcome them. He rebuked the disciples: "Let the little children come to me, and do not hinder them, for the kingdom of God belongs to such as these" (Mark 10:14). Their desire to be with Jesus and their child-like faith, Jesus was saying, should be an example to us.
Ask God to show you if He wants you to accept this responsibility, and if He does, do it with all your heart. Pray for the children in your class... ask God to help you teach them simple stories from the Bible... and be an example to them of God's love and patience. Just as Jesus welcomed the little children who came to Him, so you can welcome them in His name, helping them know God is real, and that He loves them.
You may never know the results of what you'll do, but that doesn't matter. I suspect my own Sunday School teachers shook their heads in despair at my restlessness! But God used them to plant seeds, seeds of faith in Christ, that would later sprout and bear fruit. May God bless you and use you to do the same with these children.