Does my semi-agreement to someone swearing an oath in front of me count as me having sworn an oath, for fear that I would have thereby sworn falsely?

It appears that your utterance of this word was not intended as an oath, and therefore it does not constitute a binding oath requiring expiation. If it occurred unintentionally, it falls under the category of forgivable idle talk (laghw) and does not necessitate expiation, as Allah Almighty says: "Allah will not impose blame upon you for what is unintentional in your oaths, but He will impose blame upon you for what you intended of oaths." (Al-Ma'idah: 89).