Are we sinful if we tell our mother—who is suffering from a poor psychological state and obsessive-compulsive disorder—that our brother has passed away, after we had told her he was traveling?

It is not permissible to lie in this situation, but you can resort to tawriyah (equivocation) and ma'areedh (allusions) by using general phrases from which your mother can infer your brother's well-being. For example, you could say, "So-and-so is fine," intending that he was fine yesterday, or "He traveled," intending his travel to the afterlife. Indeed, in ma'areedh, there is a way out from lying. You should also recite to her verses about patience and its reward, and the narrations of the Prophet, peace and blessings be upon him, regarding the virtue of patience upon losing a child.