What is the ruling on prayer and fasting in the aforementioned cases, especially concerning blood discharged before the expected time of menstruation, and is it considered menstrual blood or irregular bleeding (istihada)?

If the bleeding period exceeds fifteen days, then your wife is considered to have Istihadah (prolonged non-menstrual bleeding). She should refer to her previous habitual cycle, and anything beyond that is considered Istihadah. The days preceding her usual cycle are also considered Istihadah, so she must pray and fast during them. As for the prayers she missed, thinking she was menstruating, there is a difference of opinion regarding the obligation of making them up (Qada'). The majority of scholars hold the view that making them up is obligatory, while Ibn Taymiyyah believes it is not required. Making them up out of caution is preferable.