Does my husband suffer from a mental illness, jinn possession, or sorcery, given that he acts strangely, such as talking to himself, imagining a woman is present, and masturbating, and he improves with medication then relapses?

Most likely the husband is afflicted by jinn possession. Therefore, it is recommended to recite Surat Al-Baqarah in the house, because "Satan flees from the house in which Surat Al-Baqarah is recited." Then, it is recommended to seek help from a raqi (exorcist) who is skilled in performing ruqyah (exorcism) with the Book of Allah and what has been authentically reported from the Messenger of Allah, may Allah's prayers and peace be upon him, to diagnose his condition and treat it.