Is it permissible to compel the eldest brother to relinquish his share of the house in exchange for a sum of money, and for the father to create fictitious rental contracts for the rest of his siblings out of fear of future problems, and will the father incur sin in the afterlife because of this?

Given the complexity of the issue, the lack of a clear ruling, and the fact that the father did not disinherit his eldest son, but rather set his share to the extent of his portion of the house's rent out of fear of problems, this paves the way for his excuse. We hope for his forgiveness and pardon. You should seek forgiveness for him and ask for mercy for him. Strive to reconcile with your brother on something you can agree upon to clear your father's conscience. As for compelling your brother to sell his share, this must be done through a judge. You have no means to do so yourselves without his consent. What we suggest is that you reconcile with him and try to persuade him to agree to the sale, such as through mediation, or by increasing the price of his share, or by offering him the choice between buying your shares or selling his own.