What is the ruling on saying "By Allah, I will not speak to you" to someone in order for him to return an item he took, knowing that he returned it later?

If the apparent meaning of the oath requires refraining from speaking to a person the moment they leave the room, and there was no intention or reason to restrict the absolute nature of the oath, then the speaker has broken their oath and atonement for the oath (kaffarat al-yameen) becomes obligatory upon them.