Is it forbidden for the husband or wife to remember another person after marriage, even in terms of memories?

Allah has forgiven this nation for whatever they ponder in their souls, as long as they do not act upon it or speak of it. A person is not held accountable for mere self-talk unless they indulge in it and become content with it. Remembering a person does not entail accountability, but these thoughts should be repelled before they become an idea, then an action, then a habit.