What is the reason for the discharge of intermittent brown spotting, what is its treatment, and is prayer obligatory with it?

Brown discharge is not considered menstruation unless it occurs during the usual time of one's period and its total duration reaches a day and a night, which is the minimum duration for menstruation according to the majority of scholars. If it is not considered menstruation, then you should fast, pray, and do everything that ritually pure women do. You should also purify yourself from this brownish discharge because it is impure.