I can understand why you might want to avoid this event; after all, no one likes to be caught up in an unpleasant situation. When we face hard decisions (no matter what they are), it's always tempting to take the easy way out -- which in this case probably would be to stay away.
But the easiest path isn't always the right path, and I hope you'll seek God's will in this matter. For one thing, staying away might create misunderstanding (and even conflict) between you and your hosts and others in your family. After all, you don't even know if those who might cause problems will be there.
But a more important issue is this: God may want you to attend -- not just to enjoy seeing your family, but to be a peacemaker. The Bible says, "Let us therefore make every effort to do what leads to peace" (Romans 14:19). I don't know how God might use you, but by showing love to those who are hard to love, and by refusing to get swept into conversations that might lead to controversy, you can bring peace to this gathering. Your example could help others realize the importance of family harmony.
Suggest to your hosts that they plan enough activities to keep people occupied. They also might collect old photos from family members that could bring back pleasant memories (and provide amusement!). Most of all, surround your plans with prayer, and ask God to help you bring your family closer -- to each other, and to Christ.