Query when tag was added to contact?

One of our operators fat fingered and added a tag to half our users. I’m trying to undo this by querying the API to see when this particular tag was added to a user and if it’s within some time window, I want to remove it.

However, when I look at the tags for a particular user, it doesn’t tell me when that tag was applied to a user. Does anyone know a solution for this?