Is it permissible to withhold the share of a sister's inheritance from the sale of an apartment that belongs to her but has not yet been divided, in order to offset the mother's rightful share of the inheritance?

None of the heirs may prevent the division of the inheritance, and it cannot be divided until the deceased's debt has been paid. If your deceased sister had a debt pertaining to her, it must be paid before dividing the estate, in accordance with the Almighty's saying: "after any legacy she may have bequeathed or any debt." However, if the debt is on the husband, it has no relation to the wife's estate. If your sister had money from the revenue of a building or from the sale of an apartment, her husband and children must be informed and invited to divide the estate and give the mother her right. If they refuse or procrastinate, it is permissible to seize this money and perform an offset equivalent to the mother's share from the deceased sister's estate; this is based on the issue of "al-Zafar," whereby it is permissible for the wronged person to take his right if he seizes the money of his oppressor.