What is the ruling on the oath that my brother made me swear upon the Mus'haf, that I would not speak to a person who wishes to marry me, and how can it be annulled?

If the oath was taken under duress that is legally recognized, then your oath is not binding, and no expiation is required of you. However, if you were not under duress, then expiation for the oath becomes obligatory when the sworn-upon action is performed.

As for breaking an oath (al-hant), its ruling varies depending on what was sworn upon: if speaking to the man involves engaging in something forbidden, then it is forbidden. If a benefit depends on it, then it is permissible. The ruling on breaking an oath varies between being obligatory, recommended, disliked, permissible, or forbidden, depending on the type of oath (whether it concerns an obligation, a forbidden act, a recommended act, a disliked act, or a permissible act).