Is it permissible to reduce visits to an annoying relative, neighbor, or colleague, and to limit interactions to necessary occasions, in order to avoid their harm and envy, especially given that their annoyance increases when one tries to get closer to them?

Allah Almighty has commanded us to be good to relatives, neighbors, and colleagues. The Sharia (definition) has commanded us to maintain kinship ties even with those who cut us off or treat us badly. Maintaining kinship ties does not have a specific method or fixed times; rather, it depends on custom. If visiting some relatives causes you harm, you are not obligated to visit them. You can maintain ties with them by calling them, inquiring about them, or reducing visits. As for neighbors and colleagues, you are not obligated to visit them except for condolences or congratulations; rather, what is required is to refrain from harming them.