What is the ruling regarding the ghusl (ritual bath) and prayers performed by someone who practiced masturbation in their youth without knowledge of the rulings concerning janabah (state of ritual impurity) and ritual purity?

Ghusl Janabah (ritual bath after sexual impurity) is not valid without intention. Your ghusl without the intention of janabah is not considered valid. As for the prayers you performed before the sufficient ghusl, there is a difference of opinion among scholars regarding the necessity of re-performing them. The view of re-performing them (qada') is more cautious and absolves one's responsibility, and it is the view of the majority of scholars.