Is it better to join the congregation in the mosque when it has already started, or to pray in congregation from the beginning outside the mosque? And does this preference differ among the madhhabs?

The best is to pray in congregation in the mosque, even if one misses part of it, due to the great virtue of the mosque. This is the view of the Shafi'is and the Hanbalis. So, catching part of the congregation in the mosque is better than catching all of it outside. This is supported by the opinion of those who deem congregational prayer in the mosque obligatory.