Is it permissible to continue a relationship with a fiancé that involves unlawful physical encounters, while repeatedly repenting and returning to the sin? And in this case, would the Istikhara prayer be valid?

The Shariah has blocked paths leading to indecencies, warning against mingling and seclusion with a non-mahram (definition) woman. A suitor is considered a non-mahram until the marriage contract is concluded. Therefore, your going out and meeting him before the contract is forbidden. Fear Allah and refrain from meeting him until the legal marriage contract is done, and be frank with him about this. Trust in Allah and make abundant supplications, remembering His (the Almighty's) saying: (O you who have believed, do not follow the footsteps of Satan. And whoever follows the footsteps of Satan - indeed, he enjoins immorality and wrongdoing). Be deliberate concerning this prospective husband and investigate him. Look for the obstacles and problems that are hindering the marriage with your father, and it is permissible to seek help from someone who can influence him to initiate and expedite the marriage contract.