How can I deal with my father, who was not present in my life and now wants me to visit him, given my mother's refusal and my own discomfort with his actions?

It is not permissible for the mother to prevent her children from visiting their father and maintaining ties with him, and she is not to be obeyed in that. Telephone communication does not suffice for a visit, and the father's shortcomings do not nullify the duty of honoring him. In case of fear from the father, it is not permissible to be alone with him, but the children should visit him and advise their siblings to maintain ties with him. As for internal dislike, it has its own specific ruling.