Is the pilgrim, Khalifa, obligated to compensate the amount that was handed to a person impersonating the son of Yusuf, or is the amount considered lost, and the pilgrim is not liable for anything?

You entrusted your friend with delivering a sum of money to your family at their home, and you authorized him to delegate his father, both of which are permissible. The general principle is that an agent is trustworthy and does not guarantee unless there is transgression or negligence. Your friend was negligent by violating your instruction and condition to deliver the money to your home, which led to its loss. This obligates him to guarantee the money unless you waive it for him. As for your friend, he can claim the money from his father if the father was negligent, and his negligence here is clear by not verifying the person to whom he gave the money and not having a witness for it.