Does the divorced husband of the sister have the right to compel the family of the divorced sister to host his children overnight in their home without their permission and consent, especially on a day designated for a family gathering, thereby causing them psychological harm?

You are not obligated to house your sister with you, but if you do, it is an act of kindness for which you will be rewarded. Her ex-husband must enable her to see her children at a time and place they agree upon. You are not obligated to host the children in your home, and he should understand that. If you permit them to come on a specific day, they must adhere to it, as entering your home without your permission is not permissible, for Allah Almighty says: ﴿O you who have believed, do not enter houses other than your own houses until you ascertain intimacy and greet their people.﴾, and the Prophet, peace and blessings be upon him, said: "The property of a person is not lawful except with his willing consent," and this includes benefiting from a person's home.