Is it permissible to plug the anus of the deceased female with cotton if stool continues to exit during her ritual washing?

What you did is correct, and no expiation is required of you. The emission of impurities from the deceased after their ritual washing does not necessitate re-washing them or re-performing their ablution. If the emission of impurities from the deceased does not stop, the place from which they exit should be plugged with cotton. If that does not hold, then a tight clay plug should be used to prevent the impurities from coming out.