What are the consequences for a person who paid money to his relative to kill his brother's murderer, who was not sentenced to death, knowing that the second murderer has children who became orphans, and that the instigator has repented to Allah?

What that person did is forbidden, as it involves taking jahiliyyah (pre-Islamic) vengeance and killing a soul that Allah has forbidden to kill except by right. Furthermore, Allah's saying: "And no bearer of burdens will bear the burden of another" (Quran 35:18) indicates that no one should be held accountable for another's sin. Even if a judge rules for retribution (qisas), it is not for individuals to carry out; rather, it is the prerogative of those in authority.

The killer must seek forgiveness and sincere repentance, multiply good deeds, and show kindness to the children of the deceased, bringing joy to them.

As for the consequences of the killing, whether retribution, blood money (diyah), or expiation (kaffarah), they fall upon the one who directly committed the act, not on the one who ordered it. This is based on the jurisprudential rule: "If the direct perpetrator and the cause converge, liability falls on the direct perpetrator."