Is shyness about praying in front of people, and abandoning an obligatory prayer because of that, considered minor or major Shirk, and does prayer become invalid with this feeling, even if performed at home?

As for abandoning prayer out of shyness before people, it is a grave evil and a serious sin. As for praying in the presence of people while feeling shy of them, it is something that is not befitting. It does not fall under ostentation (riya') or polytheism (shirk), but it is a blameworthy feeling. It appears that this feeling stems from a psychological ailment. However, prayer is not invalidated merely by the feeling of shyness during its performance.