Would a mother's prayer be considered invalid if she had been performing the prayer after the opening Takbeer for many years?

What your mother did was a grave mistake due to her negligence in learning the rulings of prayer, and she must seek forgiveness. As for her past prayers, they are valid, because the preponderant opinion is that the words of one who forgets or is ignorant during prayer do not invalidate it, based on the Almighty's saying: "Our Lord, do not take us to task if we forget or make a mistake," and because the Prophet, peace and blessings be upon him, did not order Mu'awiyah ibn al-Hakam to repeat his prayer when he spoke during it out of ignorance.