Back in November I blogged about the biggest language and personality study ever undertaken and the fact that researchers confirmed they were able to assess your personality by scanning your Facebook likes and status updates. At the time, the research findings got very little coverage and those that did hear about it were very sceptical. I said I would follow this theme on my blog and so, a little later than originally planned, an update!
Last summer, Marc Mapes, updated me on his latest adventure – eiTalent which is, co-incidentally a values based applicant screening technology start up that used very similar natural language based assessment techniques to improve applicant screening accuracy, based purely on the contents of the CV. Yep, thats right. Simply scanning the text included on the CV was, he said, good enough to be able to identify candidates who would be a good ‘fit’ for the organisation and with highest potential to succeed in role.
Now this was news. My day job sees me explore the notion of predicting potential. It’s our bread and butter at Chemistry. In doing so we use traditional and proven assessment techniques yet here, hot on the heels of the PLOS research, was another organisation using non structured language techniques in assessment. I caught up with Marc recently to see how they were doing and he ran me through the case study that put them on the map:
“We worked with a major retailer and as a starting point identified 10 key values across the business. The retail resourcing team gave us 200 anonymised CV’s, broken into 4 groups:
- 50 high performers who had been promoted within the first 12 months of being hired,
- 50 poor performers who had been fired,
- 50 CV’s of individuals who were interviewed but not hired due to cultural fit
- 50 random CV’s of candidates that were not hired
The CV’s were not identified to us in any way. In our first analysis we were able to identify 37 of the 50 high performers. Not bad, but not great. Further analysis showed that the CV’s that we missed contained only names of employers and dates. The narrative was missing which is a key element. The exercise was monitored by an independent psychologist and our technology was scrutinised by the clients own data science team. We won the deal. In our next client, a major law firm, we were able to improve our first pass strike rate to 87%. We won that deal too.“
Since our first conversation last summer, Marc and his crew have already extended functionality to include LinkedIn profiles and will soon be adding… yep you got it, social profiles including Facebook and Twitter. They also plan to include video interview transcripts and email.
This has huge implications for the resourcing technology industry. Over the last 15 – 20 years the recruitment software industry has poured £m’s into developing tools that can scrape CV’s and LinkedIn profiles to identify individuals with the most relevant ‘key skills and experience’. Whether it be the ATS providers, specialists like the ill fated “iProfile” or pure plays like Daxtra, these guys try to outdo each other by making their technology ever more sophisticated, identifying key words and phrases and contextualising them against dates and job titles, all in an attempt to convince you that this tech is so smart, it will deliver you the perfect candidate from the volume of crap that resides in your ATS.
Turns out they have got it all wrong. The problem is that no matter how many coding geniuses you throw at this approach, its never gonna work and thats because fundamentally they are measuring the wrong thing. Previous Experience – the very focus of the programming effort – is the least reliable predictor of potential and performance in a role. What makes technology plays like eiTalent so interesting is that they focus on the language that traditional technology works so hard to eliminate from the parsing process – non work or skills related dialogue. Words and phrases that say more about who you are than what you have done. What’s important here is that these words say a lot about your personality or more specifically, give and insight into your values, motivations and likely behaviours – they key predictors of potential.
I firmly believe this kind of analysis will dictate how we assess in the future. It’s no co-incidence that a public form of the work based on the original PLOS research has emerged – see www.labs.five.com. They are going offline temporarily on the 20th of July but watch out for more from them later in the year.
In the meantime, if you are in the resourcing technology arena take a tip from me and start looking at the potential of unstructured social content before its too late. Mark my words, the days of sifting candidates by skills and experience are numbered.
*Disclaimer – Just to be clear I do not accept brown envelopes full of cash, cuddly toys, free massages or any other form of bribe or incentive to write about technology or specific technology companies so my thoughts and views are completely my own 😉