Does refusing to go with one's father and brother to visit relatives invalidate sincere repentance, especially if the refusal angers them, and one fears it falls under the category of "angering people," which may not be forgiven?

Congratulations on your repentance. Visiting relatives is an act of worship for which one is rewarded, as long as no religious prohibition occurs. It does not prevent the acceptance of repentance; rather, it brings joy and forgiveness of sins through shaking hands. One must beware of shirk (definition) (associating partners with God) without being overly scrupulous, and recite the supplication: "O Allah, I seek refuge in You from associating partners with You knowingly, and I seek Your forgiveness for what I do not know."