How can the prohibition against pessimism be reconciled with the hadiths that describe this worldly life as a prison for the believer, or that call for living in it as a stranger or a wayfarer, especially when these hadiths conflict with a person's desire for joy?

We do not know of a Hadith with the wording "Do not be pessimistic." Pessimism is not legislated and is considered a bad opinion of Allah the Almighty. A Muslim must always be optimistic because pessimism is a bad opinion of Allah without cause, while optimism is a good opinion of Him. As for the Hadiths that encourage asceticism in this world, they do not contradict having a good opinion of Allah, nor do they contradict a controlled joy that adheres to the limits of Islamic law and enjoying permissible things, for Allah the Almighty says: (But seek, through that which Allah has given you, the home of the Hereafter; and [yet], do not forget your share of the world), and His saying: (Our Lord, give us in this world [that which is] good and in the Hereafter [that which is] good and protect us from the punishment of the Fire).