Are clothes that have touched an impurity without washing the hands rendered impure, and must I inform my brother who prayed in them?

Waswasah (obsessive whispering) is a insidious disease. What you mentioned about your hand touching your clothes, and then your brother's clothes, appears to be one of its symptoms. The default assumption is that your hand was not impure when you touched your clothes with it. If there is doubt about its impurity, then the default is to discard and disregard this doubt, and not act upon it, especially in your case. You are not obliged to inform your brother of anything, because the default assumption for his clothes is that they are pure. You are not responsible for invalidating his prayer because it is not invalid. You must completely and entirely turn away from these whispers.