Is it permissible for me to complete the prayer with another congregation when I have caught the Imam in the last رکعه, and there is already an existing congregation in the mosque?

The prayer of those who follow a latecomer (masbuq) is valid, although it is better for them not to do so for two reasons: First, the validity of following a latecomer is a point of disagreement among jurists. Second, it leads to multiple congregations in a single mosque, which is a manifestation of disunity. As for the latecomer, there is no blame on him because he intended to complete his prayer individually.