Is the driver who swerved his vehicle and caused damage to my car fully financially liable, or do I bear part of the responsibility because I was backing up slowly, despite his admission of fault?

In this incident, the liability falls upon the direct cause of the injury. So, if the damage to your car was due to the other driver moving forward, then the liability is on him. However, if the damage was due to your driving forward while he was stationary, then the liability is on you. It should be noted that traffic law is the arbiter in such matters, unless it contradicts Sharia. If the other party willingly agrees to pay compensation, then it is permissible for you to accept it.