Is it permissible to take a small interest that is deposited at the end of the year into a current account opened in a bank on the basis that it is interest-free, given the argument that the enquirer did not stipulate interest to the bank and contracted with it not to take it?

Usurious interest is not permissible, whether its owner is aware of it or ignorant, and whether it is small or large. Ignorance only removes the sin, and what is obligatory is to dispose of these interests by spending them on charitable causes. It is also obligatory to withdraw money from interest-based accounts, and if you wish to keep money in banks, it should be in a bank that operates according to the provisions of Islamic law, because there is no excuse for a Muslim to deal with interest-based banks when an Islamic alternative is available.