Is it obligatory to repeat the prayers that were performed behind an Imam who did not pray with tranquility, knowing that separation was obligatory but the worshiper was unaware of that at the time?

If the Imam performs the prayer with the sufficient degree of tranquility (ṭumaʾnīnah), then his prayer is valid, and the prayer of the congregants is also valid.

However, if he does not achieve the sufficient degree of tranquility, and the congregant believes his prayer to be invalid, then it is obligatory for the congregant to separate from him. Otherwise, the congregant's prayer becomes invalid, and he must make it up.

But if the Imam is following an opinion that does not consider tranquility a pillar (rukn), and the congregant prays behind him, then the congregant's prayer is valid, provided that the congregant himself has achieved tranquility.