Should the Imam vocalize the Basmalah and recite certain Surahs that the worshippers are accustomed to, or should he relinquish the Imamate if that causes discord?

Firstly: Uttering the Basmalah aloud in prayer is permissible, not an innovation (bid'ah) or forbidden. However, it is preferable to recite it silently, as the Prophet (peace be upon him), Abu Bakr, and Umar used to begin their prayers with "Alhamdulillah Rabbil-Alameen" (All praise is due to Allah, Lord of the worlds) and did not utter the Basmalah aloud. This is the view of the Hanafis and Hanbalis, while the Shafi'is consider it Sunnah to utter it aloud. Nevertheless, there is no harm in uttering it aloud, especially for those who hold this view, to win their hearts. Leaving what is preferable for what is permissible is allowed for a legitimate Islamic interest, such as unifying hearts and teaching the congregants.

Secondly: Persisting in reciting specific surahs and verses in particular prayers by way of تخصيص (designation), if the Shari'ah did not establish such persistence (e.g., always reciting the end of Surat Al-Baqarah in Maghrib prayer), then the Sunnah is not to persist; this is to avoid contradicting the Prophet's (peace be upon him) practice of diversifying recitation and to prevent the ignorant from believing it is recommended. However, it is permissible to recite what the worshippers are accustomed to and to persist in it for a period, to achieve a greater good, which is the unity of their word and their harmony, and to endear the Sunnah and the scientific methodology to them.