What should I do if I owe an expiatory oath (kaffarat yameen) but have forgotten what I broke it with? Is it permissible to go for Umrah with my mother and brother and stay in Mecca for ten days to perform more than one Umrah, then return with only my mother and I, along with my sister and her husband?

Regarding expiation for an oath, expiation becomes obligatory for you if you are certain that you made an oath and then broke it, and it was an oath for which expiation is required. It is not a condition to remember the specific oath itself.

As for returning without a mahram (definition), it is not permissible. Since you traveled with your brother, you must return with him or with another mahram. It is not permissible to return with your sister's husband because he is not a mahram to you.