Are the insistent thoughts and questions that come to mind about glorifying Allah, praise be to Him, considered disbelief (kufr)? And can one get rid of them, knowing that they cause severe fear and anxiety?

What you are complaining about is merely waswasah (whispering of Satan), and disbelief is not incurred by it as long as the person dislikes it. A servant's dislike of these satanic thoughts is a sign of sound belief and strong faith, as indicated by the hadith: "That is pure faith." The verse "Make no excuses; you have disbelieved after your belief" does not apply to those afflicted by waswasah, but rather to those who mock religious rituals. Discard these whispers, make abundant supplication and remembrance [of Allah], and occupy yourself with what benefits your religion and your worldly life.