What is the ruling on fasting and prayer, with doubt as to whether menstrual blood descended before or after breaking the fast, and after completing the obligatory prayers?

The menstruation mentioned is not considered to have occurred except when observed, as long as you doubt its occurrence before then. This is because the default is to attribute an event to its closest time. Therefore, all prayers and compensatory fasting prior to the later of the two times are valid.