How can one reconcile the Prophet's (peace be upon him) condemnation of those who openly commit sins with the laughter of the Companions when matters of the Jahiliyyah were mentioned, and the Prophet's (peace be upon him) smiling at that?

Openly committing sins and speaking about them is forbidden, based on the hadith from Abu Hurairah: "All of my Ummah are pardoned except for those who commit sins openly." This does not contradict the hadith from Jabir ibn Samurah in Sahih Muslim that the Companions used to talk about matters of the pre-Islamic era (Jahiliyyah) and laugh, because the open sinning of others (non-Companions) is based on being pleased with the sin, being attached to it, or boasting about it. As for the Companions' discussions, they were a way of mentioning the blessing of Islam upon them and how they were brought out of misguidance into guidance, as is clear from the stories they used to narrate.