Would the questioner be sinful for refraining from accompanying a person who insists on asking him to buy food and sweets, and in whose behavior he sees ill manners and lack of chivalry?

You do not sin by not going with him. You should advise him and remind him of the hadith (definition): "And interact with people with good character," for his action is not of good character. Advise him to stop asking people [for money/favors] because it is contrary to chivalry (Muru'ah). If he responds positively, then that is good. Otherwise, "a beautiful abandonment is better than a harmful association." And beautiful abandonment is abandonment without harm, while still giving him his due of greeting him with peace and limited interaction without close association.