Many churches have nurseries for babies and young children; if yours doesn't, you might volunteer to help get one started. Other parents would probably appreciate it also.
However, don't conclude that children never get anything out of church. No, they might not get much out of the sermon, but they'll have an opportunity to learn hymns, and they will also realize that God listens when His people pray. They'll also understand that God is real, and one reason you and the other members of your church come together each week is so you can learn about Him. Jesus promised, "Where two or three gather in my name, there am I with them" (Matthew 18:20).
Most of all, your children will learn that God is important to you. Don't underestimate the importance of your example! Telling them about Jesus is important, but just as important is demonstrating your faith in Christ by the way you live. The Bible says, "In everything set them an example by doing what is good" (Titus 2:7).
Don't, however, express your faith to them only once a week. Instead, make Christ part of your family's life every day. Before every meal, thank God for your food; take time at night to read to them from a child's Bible storybook; teach your children a few simple prayers. God has given them to you; ask Him to help you give them a strong spiritual foundation. Then, the Bible says, "we will grow to become in every respect the mature body of him who is the head, that is, Christ" (Ephesians 4:15).