Is saying "Go ask your mother who your father is" considered a slanderous accusation (qadhf) that necessitates punishment, especially if it was said in response to provocation and a claim of witnesses to that?

This statement is not an explicit accusation of slander but rather an innuendo, and scholars have differed on whether an innuendo constitutes qadhf (slanderous accusation) warranting the prescribed punishment. A judge's ruling based on false testimony is inwardly invalid because it does not alter the reality of the matter. If, by this statement, you intended to accuse of zina (unlawful sexual intercourse) and impugn the lineage of your nephew without evidence, then you have committed a great evil and a major sin. You must repent by abandoning the sin, regretting your action, resolving not to return to it, along with seeking forgiveness from the person whose right you violated.