To what extent is the hadith "He whose prayer does not restrain him from indecency and evil has no prayer" authentic, and does 'indecency' differ from 'evil'?

The hadith referred to in the question is not authentically attributed to the Prophet, peace and blessings be upon him, but it was narrated as a saying of Ibn Mas'ud, may Allah be pleased with him, attributed to him [mawquf]. Al-Munkar (the reprehensible) is what the Sharia has forbidden, encompassing all sins and vices. Al-Fahsha' (indecency/lewdness) specifically refers to adultery, and it has also been said to mean every repulsive act, whether in speech or deed. Thus, al-munkar is more general than al-fahsha'. There are authentic hadiths established from the Prophet, peace and blessings be upon him, and weak hadiths that are not established from him.