Does the cursing that came from the pilgrim to preserve the sanctity of Hajj, due to the ill-treatment they received from the driver, invalidate the Hajj?

If the driver was late due to traffic, then there is no blame on him. However, if he intentionally mistreated you by insulting or delaying without reason, then he has erred. You should have exercised self-restraint, patience, and forgiveness, in accordance with the Almighty's saying: "And they repel evil with good" and "Repel evil with that which is better." As a pilgrim, you should have been patient to attain the full reward, as the Almighty says: "There is to be no sexual intercourse, nor wickedness, nor disputing during Hajj," and the Prophet (peace be upon him) said: "Whoever performs Hajj and does not have sexual relations or commit sins, will return [free of sins] as on the day his mother gave birth to him." Your Hajj is valid, and you must repent for what emanated from you to compensate for the reward you missed.