Don’t Overlook the Unintended Consequences of Poorly Designed Assessments

Share This
Author: Andrew Bateman | Source: HCI | Published: November 6, 2014

As we were winding down another day in the open office environment affectionately known as HCI North, an informal brainstorming session had turned to the topic of unconscious bias. Our resident data scientist passed along an online assessment tool to test our own hidden biases. Once we completed the initial portion of the test designed to capture demographic data the assessment veered off into a section of rapid word associations designed to tap into the subconscious mind. After about 5 minutes I was getting kind of anxious and I said out loud, “I think my brain is about to explode!” This sentiment was echoed by a colleague who had reached the same point in the online test.

The experience of feeling frustrated as I took an informal, just-for-fun assessment got me thinking about the process that job applicants go through as they are asked by potential employers to complete a pre-hire assessment. Do their brains hurt as they fill out an assessment that is meant to measure their eligibility and suitability for a new position? Isn’t it likely that they are already a little on the stressed side as they are preparing for the major life change that comes with a new job?

To get some insight into some of these questions I looked into one of our resources on assessments, Making Sense of the Assessment Nonsense, from Harrison Assessments. The first point that jumped out at me was a series of key questions to consider – specifically their number one question:

  • Is the assessment work focused and presented in a manner that builds the confidence of applicants and employees?

In my opinion this is a really critical first step. The questions within an assessment should not only be very closely aligned to the business objectives, but also easy for the applicant to understand the reasoning behind each question. A thoughtfully designed assessment should provide clear connections between the applicant’s ability and the expectations of the job.  Without this transparency an organization runs the risk of losing applicant focus during the assessment, thus compromising two key components of a good assessment; reliability and validity.

Secondly, this notion of building the confidence of applicants and employees through assessments should not be overlooked. The results of my unconscious bias test left me admittedly disappointed- and while it’s not entirely apples to apples- I think that the frustration I experienced during the assessment played a substantial role. In contrast when I took an assessment on Personality Archetypes a few months ago and the results labeled me a “Secret Weapon” I was left with a different feeling altogether.

Finally I think it’s important to key in on the simple fact that a lengthy and intensive assessment may not be the best way to introduce your organization to prospective candidates. Back in the day I remember interviewing for a job in the financial sector as I’ve always had an interest in personal finance and I believed I could bring that passion to the right organization. The brief introductory interview went well, and afterward I was asked to take part in an 8 hour pre-hire training session. I realize now that in some industries this isn’t outside the norm – but for me, at the time a recent college graduate, this seemed a little strange. 8 hours? I would get lunch, of course, but seriously, 8 hours?  I was told the reason behind this was so that the company could get to know me and I could really get to know what it would be like to work there.  

Making Sense of the Assessment Nonsense, suggests an easier and more cost-effective method to determine job-candidates qualifications; by filling out a simple, 5-minute eligibility test at the time of application. By only investing in further assessments and screening for best fit once you’ve gotten the candidate in the interview room, you can take comfort in knowing with greater certainty that they possess the measurable qualities that you expect and the desired behaviors that will allow them to succeed.

For more on this topic I encourage readers to view the webcast featuring Suzanne Miklos, President O.E. Strategies, Inc. and featuring a case study from Cathy Street, Staff Development Director Florida Sheriffs Youth Ranches, Inc, titled To Assess or Not Assess - The Dilemma! The Pitfalls and How to Avoid Them, available on demand at hci.org.