Is it permissible to wear a new garment and pray in it after washing it, if it has brown stains tending towards reddishness, without knowing the origin of these stains?

If you are not certain that the garment has been contaminated with impurity, then the original state is purity. You may wear the garment and pray in it, and your prayer will be valid, even if the stains remain on it, as long as you are not certain that it has been contaminated with impurity.