Is obedience to parents obligatory in completing university studies, despite a lack of desire for the specialization and a wish to avoid mixing, or can they be compensated with the amount they paid to the university?

May Allah reward you for your eagerness to honor your parents and seek religious knowledge. You did well by leaving the university that involves forbidden matters, such as mixing (between genders). Obedience to parents is in matters that benefit them and do not harm the son. If you can complete the studies your father desires without harm to you, for example, through distance learning, then do so; for it will bring him joy and fulfill a legitimate purpose for him. There is no need to compensate your father financially, as his goal is your future, not money, and he is wealthy and does not need it.