You don't say how old your children are; many churches, of course, provide nursery facilities for very young children. Some pastors, I've noticed, include a brief children's sermon in the service, after which the younger children are dismissed to a special class. In other words, you and your wife may want to see what alternatives are available for your children.
If your children are somewhat older but are still too young to understand everything that's going on, I encourage you to keep them with you if possible. Take books or others materials to keep them occupied, and help them participate in the service whenever they can (such as during the singing). Remember Jesus' words: "Let the little children come to me, and do not hinder them" (Mark 10:14).
No, they may not understand everything, but they'll understand some things -- and most of all, they'll begin to understand that God is important to you and your friends. In time, they'll understand something even more important: God loves them, and He wants them to love Him in return. Can you think of anything that's more important for them to learn?
Pray for your children -- not just for their health and safety, but that some day they will give their lives to Jesus. Pray too for yourselves, that you will be good examples to them, and that they'll realize that Jesus is important to you. God gave your children to you; ask Him to help you to be faithful in fulfilling the responsibilities He's given you as parents.