Is it permissible to continue a contract with the bank that concerns the children, given that the bank used to take interest from me and now takes it from them? And would I be sinful if I were to terminate the contract with the bank, bearing the penalty?

The interest taken from the bank in exchange for depositing money, or taken by the bank from you, is usury (riba), which is forbidden by Islamic law. Transactions of this kind must be stopped.

Allah Almighty said: "O you who have believed, fear Allah and give up what remains [due to you] of interest, if you should be believers. And if you do not, then be informed of a war [against you] from Allah and His Messenger. But if you repent, you may have your principal – [thus] you do no wrong, nor are you wronged."

And the Messenger of Allah, peace and blessings be upon him, cursed the one who consumes usury, the one who feeds it to others, its scribe, and its two witnesses, indicating that they are all equal in sin.