Is it obligatory to re-perform ghusl if one doubts that semen or madhy (prostatic fluid) reached under the unwashed fingernails after the initial ghusl?

There is no need to repeat the ghusl (definition) (ritual bath) or the prayer, as the default assumption is the absence of a barrier, and no attention should be paid to whispers (of doubt). One must abandon masturbation through repentance, and fingernails should not be left long, as this is against the Sunnah (definition) and fitra (natural disposition).