Your pastor needs to know of your concern, and if this is as serious a problem as you suggest, then someone needs to alert him–not in a harsh or critical spirit, but out of a genuine concern for both him and the church. The Bible tells us to "admonish one another with all wisdom" (Colossians 3:16).
In reality, he may not even realize how much time he actually spends on this topic, or how people are reacting to it. After all, he probably knows better than anyone else what opportunities your church has to serve Christ, and he wants to encourage you to take advantage of them. But he also knows these projects will take money, and he doesn't want them to be abandoned because they don't have enough support. Neither should you.
Even Jesus' band of 12 disciples needed money to do their work–although nowhere do we read that Jesus asked His hearers for money. Apparently His followers saw their needs and did what they could to support them. The Bible mentions some prominent women, for example, who followed Jesus and "were helping to support them out of their own means" (Luke 8:3).
Pray for your pastor, and pray for wisdom and grace in handling this issue. One possibility would be for you to discuss your concern with one or two of your church's leaders, and leave it in their hands to discuss it with your pastor.