Must the conditions set by those giving charity be adhered to when donating it to a specific person or for a specific purpose, or is it permissible for the agent to dispose of it as he sees fit to whomever he deems more in need?

It is not permissible for an agent entrusted with distributing charity to give it to anyone other than the person for whom it was designated, even if he sees someone else as more deserving. If the agent violates the conditions set by the principal, he is considered to have transgressed and is liable for the money, unless the principal authorizes it. And if the charity is zakat, and the agent knows that the designated recipient is not eligible for it, he must inform the owner of the money.