How do I deal with my deceased sister's daughters who refuse to wear hijab and makeup, justifying it by saying, "No one will enter another's grave [to be judged on their behalf]"?

The father of the girls is responsible for their maintenance, education, upbringing, and obliging them to wear the legal hijab (definition) and dress modestly. As for you, you should be gentle with them and kind in advising them, using effective da'wah methods, and providing them with religious materials related to hijab and immodest display, while supplicating for their guidance and uprightness.