Does not forgiving someone who swindled and defrauded money affect the saying: "from the absence of malice towards Muslims and not envying them and forgiving them before sleep"? And is the money paid to him out of fear considered a bribe? And what is the way to repent if that is the case?

May Allah help the questioner recover her money. Not forgiving a person does not affect the acceptance of the supplication for the absence of malice and envy towards Muslims and for forgiving oppressors. This person can be excluded from forgiveness when malice and envy are eliminated before sleep. If charity is about forgiving those who wronged you, then not forgiving this person only invalidates the charity related to him. The money you paid to protect yourself and your brothers is not a bribe; rather, it is this person's unlawful consumption of wealth.