How can one avoid ostentation (riya') and self-admiration ('ujb), and is a Muslim sinful and their deeds invalidated if this feeling accompanies them?

What you have described—your dislike for people's praise, your distress by it, and your seeking refuge from it—does not indicate an intention of showing off (riya') or a desire for praise. There is nothing wrong with people's praise and commendation when there is no intention of showing off. Indeed, it may be among the immediate good tidings for the believer.