How is Generative AI revolutionising Early Careers recruitment?
Matching answers. Generic content. ‘Samey’ cover letters. Written assessments you’ve seen before. The involvement of Generative AI in your Early Careers candidates’ applications is probably hard to miss. It’s everywhere.
As more and more candidates are using Generative AI to complete their assessments, there are decisions to make about how you should respond. Should you condemn Generative AI and try and limit its use? Or can you embrace its capabilities, as your candidates are doing, and evolve your hiring process?
Generative AI in a nutshell
AI has been through many cycles of development. It’s not new but, with the introduction of machine-learning algorithms, it has taken off. Generative AI uses deep learning and artificial neural networks so that we can interact with the technology to prompt ideas and answers. Whereas traditional AI can only analyse data and tell you what it sees, Generative AI can create new content such as text, images, video and music by learning patterns and structures from existing data.
How candidates are using it
Early Careers candidates are digital natives, born and bred. Adopting new technologies comes naturally to them, and their use of Generative AI is already well-established.
For example, they’re using ChatGPT to write their CVs, as well as to craft responses for application forms and online assessment tests. They’re even using it to create scripts for responding to interview questions and to help pass verbal reasoning and coding tests.
GPT-4, released by the company behind ChatGPT, creates text that closely mimics human understanding and reasoning. It has helped students achieve passes in graduate exams like the BAR and CFA Level 1 and 2.
The implications for employers and TA leaders are profound.
Three ways businesses are reacting
When things change rapidly, it’s hard to know what to do. Some organisations are putting their head in the sand and hoping the problem of artificially generated assessment content will resolve itself. Others are panicking and considering turning off automated assessments. Many are simply unsure where to turn for solutions.
"It’s no different to using the internet – remember what everyone said about that?"
“All digital assessments will be rendered worthless.”
But some businesses are taking a more considered stance. By being thoughtful and pro-active and starting to embrace the new technology, they are defining ways to manage it.
“We need a clear policy and strategy around this.”
One way or another, a response is necessary, supported by an understanding of the wider impacts of any action you take. An evidence-based, strategic approach will help you achieve the right results.
But what does this look like?
Developing a strategic response to AI
There are three steps to follow when formulating your strategic response to Generative AI. Remember, no two businesses are the same, so the response you take needs to be right for you and the candidates you are trying to engage.
Step 1 - Define your stance
Many employers are moving away from labelling candidates who use AI in the assessment stages as ‘cheats’. After all, people have been using the internet to research and create presentations for years – and plagiarism is not always easy to spot. Still, it’s important to decide your view on to what extent using Generative AI is ‘cheating’.
It can be hard for TA teams to take a strong stance on this if you’re going to use Generative AI in your own business. If you can use it, why can’t your candidates? For example, if a candidate gives you an impressive response using ChatGPT, does that show they have used it creatively and proactively? And with ChatGPT, someone who doesn’t have strong writing skills can stand out in other ways.
Another consideration is Generative AI’s relevance to the role a candidate is applying for. Will the technology be used in that person’s day-to-day work? If so, their familiarity with Generative AI could be extremely relevant. Being able to prompt ChatGPT well could become a core skill for many roles.
Step 2 - Identify your risks
One of the main risks that Generative AI poses for recruiters is that it could allow poor candidates to ‘overperform’. It gives these individuals the ability to look better than they are, which could waste time in the hiring process and the long run if they were to get the role.
Candidates who can’t afford to pay for the latest Generative AI tools may also be disadvantaged. For example, the free version of ChatGPT (ChatGPT 3.5) tends to produce ‘average’ performance, whereas the new paid version (ChatGPT 4) could aid a candidate’s ability to over-perform.
And as ChatGPT evolves, it will inevitably improve its ability to tackle a wider range of assessment types.
Sova, an assessment partner of AMS, has recently determined the risks of Generative AI’s use for validity and fairness associated with a range of typical assessment tools.
The risk of Gen-AI on a wide range of assessment tools
ChatGPT is good at answering text-based assessments with right or wrong answers.
Performance of ChatGPT and prompting required
How well someone can Prompt ChatGPT inluences how much a candidate can improve their performance on a test.
Role complexity and verbal content load
Role assessment complexity and verbal content load are linked to risk levels.
Step 3 - Evolve your AI use strategy: policy for TA
With your stance and risks thought through, you can start to develop your response.
Consult the experts and look at the evidence on the table. Talk to:
• Your data science team
• IT and strategy ops
• HR
• Your workforce planning teams
• Legal and compliance
Next, determine what’s important within your business. Consider things like:
• Risk appetite. How much risk is the business willing to take on? What is the impact on review processes and fairness for all? Plan how you could mediate this during later stages.
• Regional variation. Will your AI approach be consistent across both local and global markets? Are there considerations to be made for different languages and platforms?
• Use in the workplace. Do you see Generative AI playing a part in the future of your business? What level of acceptance will you agree on for using AI in daily tasks? For example, using it to create simple routine reports or content pieces?
• Security. Generative AI poses some key security risks like data security, model security, bias and fairness, and privacy issues. How are you planning to manage these?
Lastly, communicate what you decide across all teams. It’s important that everyone understands your organisation’s stance and that Generative AI can be adopted safely and in a way that’s not detrimental to the business. Share and communicate:
• Policy and procedures.
• Rules.
• Guidelines.
Evolving your assessment design
While developing your strategic response to the role of Generative AI in your business, there are some practical actions you can take across different stages of the candidate assessment process to help manage and mediate the impact of its inevitable use. Adapt these to the policy you choose and to your organisation’s needs.
Depending on the approach you take, not all these actions will be relevant:
Inclusivity is key
Whichever tactics you adopt, it is crucial that you consider the impact that Generative AI may have during assessment stages for diverse candidates, including neurodiverse candidates or candidates who need additional support. To ensure your processes are fully inclusive, your AI policy should be reviewed on a case-by-case basis when required, and assessment data monitored to ensure that there is no adverse impact caused by the changes you make.
How assessment vendors are responding
Like businesses, assessment vendors are responding to the rise of Generative AI in a variety of ways. Some are getting proactive, discontinuing the use of verbal reasoning assessments or replacing text-based assessments with video. Some are employing more simple tactics, such as timed testing. Others have no clear response at all yet.
But as AI technology continues to evolve and its impact on candidate assessment becomes clearer, responses from vendors will need to change rapidly, too.
AI - What's next?
As AI continues to evolve and become increasingly more sophisticated, there will be some significant implications for the way that Early Careers candidates are assessed. Here are a few key considerations for how the Early Careers market may be affected:
• Conversational interviews. To mitigate the impact of ChatGPT and other AI technology, there will be a resurgence in including face-to-face interviews. Organisations are realising the importance of the ‘human touch’ in recruitment in response to AI.
• Enhanced training. Extra training for hiring managers and assessors will be essential to understand the role of AI and how to identify its use.
• Work sample assessments. Although expensive to design, these are helpful for predicting whether a candidate can perform on the job or during a training programme, and are likely to become more common as a means of measuring capability.
• Re-designed screening assessments. Assessments will move away from verbal reasoning and simple situational formats, and re-focus on blended, behavioural and motivational items. Early Careers hiring will always require candidate assessments, but the look and feel will need to be adapted.
• More in-person assessment. The risks of candidates using Generative AI are greatly reduced in live, in-person assessments. Many businesses are choosing to pivot back to face-to-face events to enhance candidate engagement and experience, and ensure that they are really getting to know more about candidates.
Embrace the AI revolution
Organisations cannot simply ignore AI. Doing so poses a huge risk, not only to the way Early Careers candidates are assessed, but also to the future evolution of your whole recruitment process. Talent leaders who do not engage with AI now will ultimately be left behind.
Over the next few years, more issues and challenges will come to light, and it’s best to be involved in the conversation now. Embracing Generative AI, rather than fearing it, and taking a considered and evidence-based approach to your response to it, will help your organisation get ahead of the curve with re-designed assessment criteria for Early Careers talent. With the right approach, Generative AI can be a useful tool for us all.
AMS Early Careers assessment solutions
Need help in navigating this new era in assessment? AMS can help you develop a clear and strategic response to AI that’s tailored to your business’ needs.
We can support you in re-shaping your assessment tools and technology, align them with your strategy, and ensure you continue to select the best candidates – fairly and efficiently.
Built with Shorthand