Am I sinful if I do not wean my son, after I told him, "By God, I will not let him go until you let him go," and I intended the statement as a mere saying and not an oath?

If you did not intend to swear an oath regarding not weaning, then this is considered a 'laghw yamin' (idle oath) and does not require expiation according to most scholars. Consequently, nothing is binding upon you when you wean the child. It is permissible to wean the child off formula milk if there is no harm to them.