Is it forbidden for a man to touch his penis or anus to help himself sleep without ejaculating?

Allah has made it obligatory to guard one's private parts, except with one's wife or what one's right hand possesses. What you do by rubbing the penis is a pretext for the forbidden masturbation, and it is considered an assault even if semen is not discharged, because it invites its discharge.

As for touching the buttocks, if it is for pleasure, then it is impermissible and a form of sodomy. If it is a habit without inserting a finger into the anus, then there may be no harm in it, but the correct thing is to abandon it because it may lead to what is impermissible.

You must strive against your soul to abandon this bad habit, and seek refuge in Allah Almighty to turn it away from you.