Is it permissible to count the remaining amount of a debt that the debtor has not paid from the zakat of my money?

You are not permitted to offset the remaining amount of your debt owed by your colleague against your Zakat. Zakat is an act of worship with specific conditions for its obligation and performance. It is not permissible to waive a debt from an insolvent person in exchange for Zakat, as Ibn Taymiyyah mentioned that waiving a debt from an insolvent person does not suffice for Zakat on cash assets, without any dispute.