No one likes to be embarrassed, and that's why most of us feel nervous when we're in a new situation -- whether it's in a new church, or a new job, or even our first day in school. And yes, you'll get over these feelings, once things become familiar to you -- just as you've done in other new situations.
In reality, though, you shouldn't be so concerned about this, because in church you're surrounded by people who love you and genuinely want to help you -- even if you don't yet know them. And that's what I encourage you to do -- get to know others in your church. Even talking casually to someone sitting next to you can make you feel at home -- and may encourage them also.
In other words, take the initiative to reach out to others in this church and get acquainted with them. Introduce yourself to the pastor, and seek opportunities to volunteer or join a Bible study. The Bible says, "May the Lord make your love increase and overflow for each other and for everyone else" (1 Thessalonians 3:12).
Elsewhere in your letter you mention that you never went to church as a child. But God loves you, and He has put within you a desire to know Him. Thank Him for this, and begin your spiritual journey by asking Jesus Christ to come into your life. Then seek to grow spiritually through your fellowship with other believers.