Does the administrator of a WhatsApp group bear the sin for the abominations published therein if he expels the perpetrator?

You have done well in preventing songs and images, and your deletion of what is forbidden is sufficient to lift the sin from you, for Allah does not burden a soul beyond its capacity. You must delete the objectionable content. As for expelling offenders from the group, that depends on what serves the greater good and what causes harm. If you hope for their guidance, keep them and suffice with deleting their transgressions. But if you do not hope for that, and their presence causes harm, then expel them to remove their evil.