Is it valid for one to exit the state of Ihram, with the fidyah remaining due and to be paid at any time, or is it impermissible for him to contract a marriage or do anything else if he performed Hajj and owes a fidyah sacrifice of a sheep which he has delayed?

Hajj has two stages of Tahalul (exiting the state of ihram):

The first occurs with the throwing of the pebbles, shaving/shortening the hair, and performing Tawaf, according to the majority of scholars. Some scholars hold that it occurs simply by throwing the Jamrat al-Aqabah. With this, everything becomes permissible except for marriage.

The second occurs upon the completion of the third ritual of Hajj, and with this, everything becomes permissible for the pilgrim, including women.

As for the blood sacrifice (dam) required for abandoning an obligatory act or committing a prohibited act, it does not affect the Tahalul. One must hasten to slaughter it. However, the sacrificial animal of one who is prevented from completing Hajj (hady al-muhsar) is an exception; Tahalul is contingent upon it.