Is not hitting one's brother in the future considered a broken oath if the intention of the oath was only to hit him at that moment?

The basis of an oath is the intention of the one who swears it. So if your intention with your oath was that if he were to leave at that very moment and he did not leave then, then there is no expiation upon you, even if he left afterwards and did what you swore about.