Does a divorce take place and is it counted if the husband says to his wife: "Divorce be upon me if you leave the house, you will not enter it again as my wife," and then her younger brother takes her out with a power of attorney from her elder brother?

The reference point for an oath – whether by Allah or by divorce – is the intention of the one who swears it. If there is no intention, then the reference point is the reason that prompted the oath. If there is no reason, then it is the apparent meaning of the wording.

Therefore, if you intended by your oath that if your wife leaves the house for her family's house, she will not return while she is under your marital bond, meaning she is divorced; then her divorce takes place. And if this divorce does not complete the three divorces, you may reconcile with your wife during her waiting period ('iddah).

However, if you intended by your oath to specifically prevent her from going out with the middle brother, or from going out only on the day of the oath; then you have not broken your oath, and your divorce has not taken place if she returned home.