Yes, Jesus was talking about heaven when He spoke these words, and He actually used this expression on several occasions. In Matthew, for example, He told His disciples that "many who are first will be last, and many who are last will be first" (Matthew 19:30).
Jesus spoke this immediately after a very wealthy young man came to Him and asked what he needed to do to inherit eternal life. As Jesus talked with him, it became clear that the man was not only rich and powerful, but also very moral and religious. When Jesus listed many of the Ten Commandments, the man could truthfully reply, "All these I have kept" (Matthew 19:20).
But Jesus looked beneath the surface of the man's life and saw what was going on in his heart: He was driven by greed and covetousness. When Jesus challenged him to give up his wealth and become a disciple, "he went away sad, because he had great wealth" (Matthew 19:22). He had obeyed every one of the Ten Commandments except the last: "You shall not covet" (Exodus 20:17). Tragically, his wealth and power meant more to him than God.
Jesus' words are a solemn warning to us that earthly success will not get us into heaven. In fact, it can get in the way by blinding us to our own sin. Only Christ can save us, and no matter who we are, we need to give our lives to Him. Are you trusting Him alone for your salvation?