Is it obligatory for a brother to isolate himself from his family in order to apply the law of Allah and live as the Prophet and his Companions lived?

You must fear Allah, abandon sins, and adhere to what Allah has commanded. The presence of a brother who advises you is a mercy from Allah. As for the advising brother, inviting to Allah is obligatory and requires patience, gentleness, and kindness, especially with family. You should be wise in your method of inviting. The existence of sins does not prevent you from being good to them, honoring them, and continuing to invite them. Do not abandon your family and leave them to Satan. Rather, be patient and enduring. Abandon the place of disobedience, not the people, and seek assistance through supplication for their guidance.