Is it permissible to kill pigeons that have made a nest in the house, especially if they cause harm, and is anything incumbent upon me after I killed five of them without eating them?

If killing birds is to ward off their harm, then there is no blame in it, provided no other means exist to prevent their harm apart from killing. Other methods to deter them should be employed first. If the birds are owned by someone, their owner must be informed before killing them. When resorting to killing, it must be done in a way that allows for their utilization.