Must a solitary worshiper repeat the prayer if he doubted the number of prostrations in the third rak'ah, then completed the fourth rak'ah, rose for a fifth rak'ah, and then performed the prostration of forgetfulness after the tashahhud and before the salutation?

If someone doubts having omitted a pillar of prayer, such as a prostration (sujud (definition)), and is not afflicted by obsessive-compulsive disorder (waswas), then it is as if they did not perform it. If you remembered omitting a prostration before prostrating in the next rak'ah, you should have returned to perform the omitted prostration, then completed the fourth rak'ah, and performed the prostrations of forgetfulness (sujud as-sahw). However, given your ignorance of the legal ruling, and since you performed a complete rak'ah in place of the rak'ah whose prostration you doubted, what you did is sufficient, Allah willing.