Is it permissible for the factory to deal directly with the mills it brought, and sell products at reduced prices without referring back to me, thereby wasting my effort and the agreed-upon price differences?

It is permissible to agree with the manufacturer to market his products at a specific price, with the excess amount going to the marketer. This falls under the category of permissible agency. It is also permissible to purchase goods from the manufacturer and sell them to mills, provided that possession is taken before selling them, due to the Prophet's prohibition against selling goods before taking possession of them. The manufacturer is not obligated to refrain from selling directly to mills that he became aware of through the intermediary, because the agency contract is revocable by both parties, and the manufacturer is not bound to continue dealing with the intermediary or to refrain from direct dealings with the mills.