What is the ruling on taking commission for selling a plot of land, what should be done with the money if it becomes apparent that it is not permissible to take it, and is that considered a betrayal?

It appears that you were appointed as an agent to purchase land without specific designation, which implies the obligation to act in the best interest of the principal by acquiring the best land at the lowest price. However, your combination of agency and brokerage contradicts the essence of agency, as brokerage serves your personal interest, which is incompatible with the purpose of the agency and will bring accusations against you. Your purchase of the land at market price does not negate the accusation. Based on this, you must pay the money to the heirs, with or without their knowledge. It is not permissible to donate it on behalf of the deceased when it is possible to deliver it to them. You must also seek Allah's forgiveness for your falsehood against your principal.