Does adding or subtracting a raka'ah in prayer out of forgetfulness invalidate the prayer?

If a worshiper adds an extra rak'ah due to forgetfulness, their prayer is valid, and it is sunnah for them to perform the two prostrations of forgetfulness (sujud as-sahw) if they are still soon after the prayer. Some scholars hold the view that prostration is always recommended. However, if they omit a rak'ah and remember it a short time later, they should return to complete it and perform the two prostrations of forgetfulness. But if they remember it after a long time, their prayer is invalid and must be repeated.