I have a team that is measured heavily by first reply time as reported through Custom Reports, but cases will often be transferred to them. Does the previous team's reply time then count in their metrics, or does First Reply calculate from when something is assigned to a team?

The first response time refers to the very first reply from any teammate to the customer's very first message. Everything else is subsequent response time.