Is it permissible to fast for a day as a form of self-punishment for neglecting the regular Sunnah prayers or for swearing oaths in a state of anger? And is that considered an innovation (bid‘ah)?

There is no impediment to financially penalizing oneself for committing sins, as Ka'b ibn Malik did when he donated his wealth, and the Prophet, peace and blessings be upon him, approved of him keeping some of his money. There is also no impediment to punishing oneself with fasting when committing a sin, in order to discipline the self and prevent it from falling into the unlawful. This is not a reprehensible innovation. One should not pay attention to the feeling of fear regarding charity, for abandoning sins is better than committing them. The true motive for all of this is the fear of Allah, so do not pay attention to the whisperings of Satan.