Is there a sin or specific legal ruling incurred by someone who addresses his wife as "ya hurmah" (O woman/female)?

There is nothing wrong with that as long as the speaker does not mean that she is forbidden to him. The man's family is his harim because he prevents others from her and protects her. The harim is that for which one fights and protects, and it is that whose touch is forbidden, so one does not approach it.