Does saying "it is forbidden for me to go down again" after the word "forbidden" came to my mind, count as an oath or a prohibition that must be fulfilled, knowing that I did not intend thereby to prohibit my wife or divorce, and that I did not go down to buy bread at that time?

What you mentioned does not affect the sanctity of marriage; it is from the whisperings of Satan. Block this door and others against him.

The disagreement regarding the prohibition of one's wife is whether it constitutes ẓihār, ṭalāq, or an oath. As for prohibiting anyone else, it is considered idle talk by some scholars, while others say it necessitates an expiation for an oath. You should seek refuge in Allah and turn away from these whisperings.