No, you aren't being picky or narrow-minded, and I hope your husband will reconsider his attitude. Many companies, I understand, are becoming more and more concerned about employee theft, and he could even end up losing his job.
But even if that isn't likely, it still is wrong to take things that don't belong to us whether they come from our employer or from anyone else. Saying that "the company owes it to me" is just an excuse. Imagine what could happen in a large company if everyone did this; it could cost them millions of dollars (and it sometimes does). The Bible is clear: "You shall not steal" (Exodus 20:15).
But I sense a larger issue may be at stake here with your husband and that is the place God has in his life. When we allow dishonesty to creep into our lives, and we tolerate a lack of integrity in our character, it often is because we are ignoring God and His will.
Yes, your husband needs to stop taking small things from his employer but more than that, he needs to face his spiritual needs and put Christ at the center of his life. Perhaps this is true of you also. Don't ignore what God is trying to tell you, but humble yourselves before Him and commit your lives to Christ.