OPM’S COMPETITIVE SELECTION AVATARS
Last week the Washington Post reported that OPM has begun using avatars to measure the aptitude of people applying for federal jobs. The Post wrote that the system, “…uses animated avatars and videos to simulate challenges that could confront employees, testing their reasoning and problem-solving skills….a correct answer leads to a harder question and an incorrect one rachets down to an easier option.” It also revealed that “Federal employee unions said they were unfamiliar with the new hiring system and could not take a position.” Those are two very different and very important stories. First, let’s consider the test.
To date, the test is only for those seeking to get into the federal government and perhaps those current feds who want to not just apply for a vacancy through the internal promotion route but also apply as an outsider. However, if these tests prove attractive to agencies you can bet they will start inserting them into the promotion process soon. That will put them smack in the middle of the bargaining table talks and soon thereafter the mainstream of the grievance process. So, let’s chat about testing.
Testing is one of those things that only works perfectly in theory and rarely in practice. It comes with some notorious flaws, just like every selection system does. The old Civil Service Commission used a test in the 60s and 70s to measure aptitude known as the Federal Service Entrance Exam (FSEE), but had to drop it under pressure from the U.S. courts that found it to discriminate against minorities.
At least those of us sitting around the FEDSMILL editorial table settling up our betting debts after Saturday’s Final Four games and looking to make a killing on tonight’s game are OK with testing. While we like that it can be a bit more objective than the alternatives, we are thrilled that it makes it far easier to statistically analyze the impact of the test.
For example, unions should be demanding to see the following data:
- The contract with the vendors who prepare the test to see exactly what they were hired to produce and measure. Let’s get all that out in the open.
- Any data OPM and agencies have on the connection between test scores, the other assessment elements of the hiring process, and the first year performance appraisals of those hired. Unions need to know whether those who score highly with the avatars do just as well in interviews and thereafter on the job. Personnel psychologists and the courts call that validation. If those with the highest test scores do not generally get the highest performance appraisals after a year on the job, then the test is likely not worth the effort. Given that OPM has implemented the test in IRS, Border Patrol, Census and some other places, unions should also demand that the data be sortable by type of job. It may that that this is a great predictor of success for an IRS Customer Service Representative, but a terrible one for Border Patrol Agents.
- Any data showing the test scores by several demographic variables. Unions need the scores by race, gender, age, national origin, and disabilities to see if it meets the various Civil Rights Act standards. Moreover, the scores should be broken out separately for reasoning and problem-solving. The idea here is that if a test avoids an illegal impact on the protected civil rights classes it probably does not discriminate against any other group either.
- Any data showing the connection between test score and attrition after the first year of employment. It could be that those with the highest score are also those most likely to leave government the soonest. It helps no one to spend all the money hiring and training people who are predisposed to leave government as soon as they can. Better to hire applicants with slightly lower scores who are committed to a government career and will provide a long term return on the government’s investment.
Unions need to squeeze out as many imperfections as they can from the test before agency leaders propose to insert it into the internal promotion process. They need to send those same leaders currently so excited about the test that they will prefer outside tested applicants over internal promotion applicants a message that they need to curb their enthusiasm.
Now let’s turn to the other part of The Post story, namely, the fact that unions were not in a position to comment on this testing program. Maybe he caught them when those union leaders who do know about it were all out on spring break, hiding Easter eggs, or updating their Final Four pools. We hope so because the alternative is not good, namely, that they really did not know about the program or what to say about applicant testing. Modern HR management is a technical field with metrics and instruments and protocols. Union leader need to be experts in more than labor and employment law if they are to understand everything going on around them. They need staff who understand the science, technology, and culture of HR. Just as they currently follow the latest FLRA, MSPB and EEOC case decisions they need to be following the latest research and professional journal articles. If the failure of unions to respond to The Post reporter was because they just aren’t familiar with personnel testing, then that has to be fixed. This program has not been a closely held government secret. OPM and others have posted information about it for months.