The evaluation can be for hiring or for their overall performance in team.
What is the problem exactly?
If you analyze the trend from State of Testing report, it is evident that with increasing Agile adoption, centralized testing units are mostly getting dissolved and testers are now reporting to Dev/Team leads. With this change of structure, hiring new testers and evaluating those in teams naturally becomes the responsibility of the Dev/Team leads. And this is where things start to get interesting. How? Let's take a look.
Dev leads are usually excellent programmers with high passion and inclination towards typical programmer mindset. And this is very natural. Not all of the leads come with prior experience of hands-on testing, nor all are well versed with testing mindset and philosophy -- which is key differentiator between testers and programmers. The way programming excellence comes with deep understanding of programming concepts and ample hands-on experience of coding, it is quite natural that to understand testing better (and to evaluate people for testing skills) one must have equivalent expertise, or, at least, deep understanding of the subject. And this is something our industry is still lacking. But despite of this all, teams still need testers and they must be hired.