The Tavern represents those places, often actual taverns, where commoners hung out in each fief. Here you could examine and recruit NPCs for your service. The Tavern was, in effect, a "Court" for the commoners. Taverns are considered to be outside the keep. Thus even if the local keep is locked, you can still get to the local unemployed NPCs.