How can the prophetic hadith: "And if he intended to commit a bad deed but did not do it, Allah records it with Him as a complete good deed," be reconciled with the Quranic verse: "And whoever intends therein to commit deviation by wrongdoing — We will make him taste of a painful punishment"?

Combining the verse (and whoever intends [to commit] therein any wrongdoing or injustice – We will make him taste of a painful punishment) and the hadith (and whoever intended to commit a bad deed but did not do it, it will be written for him as a good deed) means that the generality of the hadith is specialized by the verse. Thus, the meaning is that one is held accountable for intending a bad deed within the Haram, but not outside the Haram, due to the severe emphasis on the gravity of transgression within the Haram of Mecca. This has been supported by Ibn al-Qayyim, Ibn Mas'ud, Al-Shinqiti, and Ibn ‘Uthaymeen.