'm probably not going to give you the answer you want to hear, but it would be wrong for you to lash out in anger and try to hurt this man and his family. I even wonder if you secretly hope his wife will turn against him and send him back into your arms, but if so, this too would be wrong.
You've probably heard the old expression that "two wrongs don't make a right" -- and it certainly applies in this situation. It was wrong in God's eyes for you to carry on this relationship, and even if it had ended up in marriage, it would have been without God's blessing. God takes the marriage vow very seriously, and although many today have lost sight of this, it's still true. Jesus said, "Therefore what God has joined together, let man not separate" (Mark 10:9).
But it also would be wrong for you to deliberately try to cause trouble in this person's family. He also was in the wrong, but hopefully he's realized his foolishness and has recommitted himself to his marriage.
The most important thing you can do, however, is to face honestly the wrong you've done, and turn to God for the forgiveness and new direction you need. God loves you; He loves you so much that He sent His Son into the world to make your forgiveness possible. Don't turn your back on Him any longer, because His way is always best.