Is it obligatory for me to repeat the Hajj after committing sins in order to perform Hajj on behalf of my brother, or can I perform Hajj on his behalf directly, and do sins committed after Hajj indicate that the Hajj was not accepted?

The questioner should have avoided sins and continued in obedience to Allah after Hajj. He must sincerely repent from looking at what Allah has forbidden. Falling into forbidden acts after Hajj does not affect its validity, nor is he required to repeat it for this reason. He is permitted to perform Hajj on behalf of his deceased brother. Committing transgressions is not proof that the Hajj is not accepted if the conditions for acceptance were met. Steadfastness after Hajj is an indication of an accepted Hajj (Hajj Mabrur). If he feared falling into forbidden acts, he should have married before Hajj. If he did not fear it, then prioritizing Hajj was the correct decision.