At this time, when a conversation or ticket is assigned to a certain team inbox, each available teammate can be assigned to the conversation or ticket.
When a team inbox is assigned to a conversation or ticket, it makes more sense that only the teammates that are a part of that inbox, can be selected to be assigned to the conversation or ticket.
Ideally a configuration option would exist to decide which behavior is best suited for your environment.