Is it permissible for my friend to perform Hajj on behalf of my deceased mother as a voluntary act, without my paying him anything, and am I obligated to pay for the sacrificial animal (hady) despite my inability to do so?

Your friend's Hajj on behalf of your mother is valid and sufficient. The Hady (sacrificial animal) for Tamattu' Hajj falls upon your friend, because if a deputy who is not hired performs Tamattu' Hajj of his own accord, and the one on whose behalf it is performed did not authorize him, then the Hady is incumbent upon him.