Does a Muslim remain a Muslim if he suffers from obsessive-compulsive disorder that compels him to curse Allah during prayer, and is this considered a mental illness requiring treatment?

What you are suffering from are illnesses that should be sought to be cured by consulting trustworthy doctors. Healing from it comes by turning away from whispers (waswas) and not paying attention to them, for they do not harm, nor do they take one out of Islam, nor do they affect the validity of prayer. One must strive to repel them and get rid of them.