For as long as employee assessments have been around, they're still largely a mystery to most employers. Sort of like witchcraft or a pagan ritual from the Middle Ages, most companies enter the assessment game with a lot of confusion and fear.
That's why we invited Plum's Caitlin MacGregor co-founder and CEO on the podcast to make sense of it all and relieve listeners from the pain assessment tests can sometimes bring. We're talkin' death of resumes, threats to employer branding and being stuck in World War II technology. Don't worry. It'll all make sense, and you'll feel a lot better if you're in the market for assessment solutions. Plus, Caitlin is Canadian, which means her commentary goes down as smoothly as a cold Moosehead or a warm serving of maple syrup.
You're welcome, eh.
TRANSCRIPTION SPONSORED BY: Disability Solutions partners with our clients to build best-in-class inclusion programs and reach qualified, talented individuals with disabilities of every skill, education, and experience level.
Cailtlin (0s):
Myers-Briggs has never been backed up by academics. It's never been peer reviewed.
Chad (5s):
Tell me about it. I need an applause here. I need an applause.
sfx (10s):
sexy romantic soundtrack music Really. Can you feel the tension in the air right now? I know I can. I can feel it all the way down to my plums.
INTRO (21s):
Hide your kids! Lock the doors! You're listening to HR’s most dangerous podcast. Chad Sowash and Joel Cheeseman are here to punch the recruiting industry, right where it hurts! Complete with breaking news, brash opinion and loads of snark, buckle up boys and girls, it's time for the Chad and Cheese podcast.
Joel (42s):
Oh yeah. What's up everybody. It's your favorite guilty pleasure also known as the Chad and cheese podcast on your cohost. Joel Cheeseman joined us, always the Garth to my Wayne Chad Sowash and today we are just really excited to welcome Caitlin MacGregor co-founder and CEO of Plum. Caitlin welcome to HRS most dangerous podcast.
Cailtlin (1m 6s):
Thank you so much for having me. I'm really excited.
Joel (1m 8s):
We're happy to have you! For those that don't know you, can we get a quick Twitter bio and then Chad, I'll get into the heavy company stuff.
Cailtlin (1m 15s):
So before starting Plum, I actually ran a company that helps students with learning disabilities, use technology to improve their lives. And I started using a tool like mine as my own customer, and then birthed Plum as the third company that I've started from scratch. First one, that I get to own myself and really excited that we've had some great success in the industry over the last five years.
Joel (1m 40s):
Can I give her an opportunity to tell everyone she's Canadian and she passes it up? She doesn't mention Canada once.
Cailtlin (1m 45s):
I'm Canadian. I'm a mom of two to, an 11 and six year old that are crazy full of energy, and a parent to a dog. Yeah, there's, there's more there too.
Chad (1m 58s):
11 and six and yet you're starting up a company? Wow! Okay. So let's just jump right into that. Why start an assessment organization today when we already have so many, I mean, what's wrong with what we already have out there, Kaitlin?
Cailtlin (2m 14s):
I mean, how many people right now listening love assessments. That's the problem. There's so much wrong with how assessments have been used in HR. There's so much room for improvement. And we saw that firsthand. I had to pay a very, very expensive consultant to get access to this kind of data when I was running my ad tech company. And I couldn't understand, you know, students that are unique, that needs special accommodation, that needs to be understood for who they are, have IEPs and why in the working world don't we have that kind of information that helps us understand who a person is and how to support them and set them up for success and the reason for that is that consultants make lots of money by making it really the opposite of transparent.
Cailtlin (2m 59s):
Make it very difficult to understand, make it so that you have to pay them lots of money to be able to utilize the data. And they weren't really designed for the human, that they're about. They're not really designed for the individual to empower them to be successful. And so I saw an opportunity to democratize access to this highly predictive data, because it is a science, there's an entire field of industrial organizational psychologist, but they don't understand how to use technology properly to scale it and make it accessible and make it actually help inform the decisions and create the type of world that I think everybody in HR wants to create where the human is at the center of the work that we're doing.
Chad (3m 36s):
Yeah. So, okay. Help me understand because individuals with disabilities and it sounds like that was definitely one of the core reasons why you jumped started this, but individuals with disabilities in some cases are wired just differently. And that most assessments are incredibly abelist, racist, sexist, classist. I mean, how can a system be different? How can you universally design away from everything that we've had thus far that has been all of those things and even worse in some cases?
Cailtlin (4m 5s):
Yeah. Well, first of all, can we please get rid of, I'm going to, a lot of people are going to hate me for saying this, Myers-Briggs has never been backed up by academics. It's never been peer reviewed.
Chad (4m 16s):
Tell me about it. I need to know the applause here. I need to know applause. Keep going. I'm sorry. And
Cailtlin (4m 25s):
Then after Myers-Briggs came DiSC, you know, which was developed in World War II and I appreciate what I was trying to do at the time, but my goodness, you know, it's 2022, the research has come a long way and it is absolute shame to the entire IO psychology community that that's still being such widely used. There are so many models out there that have were designed 50 and 60 years ago and people have just latched onto them and not moved on. But if you go to, you know, the research that is really about trying to figure out how do you avoid that bias? How do you make sure that you're not discriminating against certain socioeconomic groups? There's a whole bunch of research, for example, that has shown that when you use language or math questions, it does discriminate against socioeconomic groups.
Cailtlin (5m 11s):
It does discriminate against what your first language is, and it tends to prefer English speaking candidates. So there's so much research that has shown what's wrong, but there's also tons of solutions that have come out of that. There's a whole bunch of recommendations. The bar has been risen much higher. And if you talk to, you know, the people that understand this research, they're often not the ones that understand how to connect it to the business world, to connect it in a way that is accessible for other people. There's this disconnect of people in the technology field that are like, Ooh, we want something sexy. They try to create the science and they cut a lot of corners or you get the people that understand the science and they don't understand the technology and so they never get beyond that consulting service.
Cailtlin (5m 55s):
And so we saw this opportunity be a bridge between both worlds. Let's take the best that technology has to offer in terms of creating that, you know, candidate and employee experience that we all strive to have, where they get benefit and value from the process and it can scale, but also let's keep the science intact. And so many companies cut corners on all sides, Because frankly, the customers aren't educated enough to necessarily know who to trust and so they're kind of taking advantage of who has the best marketing message rather than who has the best science and the best usability and scalability.
Chad (6m 28s):
That's what I want to talk about. The educating the customer. Now here's the biggest issue that I find is it, first and foremost, everybody makes this shit so complex so that you, they feel like they need somebody and they do to some extent, but they go from a Myers-Briggs to a big five to balloon popping. To me they all seem ridiculous because I believe that there's more than 16 variations of a human being, let's just say, but when we take a look at education, it's really hard for employers to educate themselves on everything that's happening today, because there's so much tech being thrown at them. How do they know what is just total smoke and mirrors and bullshit versus what's legit?
Cailtlin (7m 11s):
Yeah, that is a great question. And something that, you know, we've been really thinking about, how do we kind of put out a best practice so that it becomes kind of a buyer's guide for people to better understand. We years ago created an ebook around this, but it's due for a refresh cause there's been lots of developments in the last few years. I think there's a couple of key elements. First of all, it does help. If you're a larger organization, normally they have access to internal expertise. I like going up against them because we know that we're going to win those deals because the first thing they look at is our scientific technical manual that backs up all of this with data, with third party input, it's everything is very explainable.
Cailtlin (7m 56s):
We do a really great job of sharing everything that we did in terms of designing this the right way from the beginning.
Chad (8m 2s):
We've got a cocky Canadian on here.
Joel (8m 4s):
Of course. Is there any other time?
Cailtlin (8m 8s):
And so part of it is that, you know, there should be a scientific technical manual that makes it very clear why certain decisions were made and it should be consistently reinforcing why they were designed with that universal design principle, meaning it was not designed for the mainstream. It was designed to be as inclusive as possible from day one. And there should be elements to that. So for us, we make sure that, you know, we're using the gold standard for personality. We also make sure that you can't fake or game it. We also make sure that we also look at cognitive, but in a way that shows that there's no adverse impact. So we look at not what you've learned in the past, like language and math, but how well do you handle brand new abstract problems? Like if you experienced something brand new, how likely are you to get to the right answer?
Cailtlin (8m 52s):
And we look at things like social intelligence, which is like <inaudible> actually scientific where it's about how well do you solve people problems cause that's important to understand for where people have their overall strengths and how to play to them, but that's only half the equation is the person and understanding what drives them and drains them. The other half is understanding the requirements for the role and that's where I think the industry has messed up in the biggest way and has the biggest opportunity to really improve, is understanding the alignment between the person and the role. So that we're screening in more people, that's what there should be about. How do we surface those hidden gems? How do we surface those people that we may have overlooked because of that past experience, because of their access to education or access to how fast they progressed in their career, how can we screen people in and service them?
Cailtlin (9m 39s):
And it comes down to what do you need in the role?
Joel (9m 41s):
I'm going to take us back to World War II again, if I may?
Cailtlin (9m 46s):
No.
Joel (9m 46s):
Let's talk resumes for a second, every so often, let's call it every week, someone decries the death of the resume should resumes die and is Plum the company to do it?
Cailtlin (9m 57s):
We have this debate all the time.
Joel (9m 59s):
Who's we?
Cailtlin (10m 0s):
My executive team and I, sorry. When I think about my kids and when they're going to enter the workforce and I think about what I want for them. I don't care if they're going to become doctors, lawyers, or teachers or nurses, I just want them to be happy, fulfilled, and thrive in their career. And so when I think about that outcome, do I need a resume to help them figure out what roles are going to make them happy, fulfilled, and help them thrive in their career? I don't think it in that vision, that end goal that we need resumes to make that happen. However, what I have come to admit more recently is that the industry's not there yet.
Cailtlin (10m 44s):
The industry is not going to be able to give up resumes entirely. So it's almost like an addiction, we need to kind of go through a long-term withdrawal. And I think the first step of that journey is to move resumes much further back in the process. You know, you don't do reference checks on every single person that applies to your company. You do reference checks near the end. I think resumes need to play a role much, much further back because if you look at the science behind, you know, what resumes are able to predict and you look at people's innate transferrable abilities, you know, people are going and we see this all with the future of work.
Cailtlin (11m 24s):
If you can upskill, if you can train somebody, what's the likelihood for them to be able to succeed? That data, which is what Plum gets at, is four times more accurate than a resume. So I want this data early as possible, screening in those people that have that alignment to the role that if you enable them, they will be successful and then say, okay, well how much training am I going to have to do? Are they gonna be able to hit the ground running? Just like you look at salary, you know, am I going to be able to afford this person or geography, are they in the time zone that I need? Those questions come later, once you're paying attention to the right people, because then you might say, Hey, this person is an incredible match, but they're going to take longer to get up to speed, but I didn't have to pay a recruiter $20,000 to get them, so maybe I'm going to give that person some of those training dollars.
Cailtlin (12m 10s):
Maybe I can't access talent that can hit the ground running right away. So maybe those are compromises I'm okay with, but other roles you need somebody hitting the ground running tomorrow.
Joel (12m 19s):
And killing resumes always feels like a luxury that not every company can enjoy. So when I think about smaller companies SMBs, you know, getting rid of the resume, I find that really challenging to get my head around. But I also think at least my perception is that most small businesses don't use assessments. Am I correct? In that opinion? And if you were talking to an SMB on this podcast, what would you tell them to convince them to start leveraging assessments and they're hiring?
Cailtlin (12m 46s):
So we started as our own first customer and we were an SMB and we started the first few years with SMBs now with eliminating resumes. You're absolutely right. It's has a profound impact when you're an enterprise company. So Scotia Bank, for example, completely eliminated resumes for all their campus hires, early entry internships and instead you apply with your Plum profile.
Joel (13m 8s):
Wow.
Cailtlin (13m 12s):
What's incredible about that story is the impact of it. I'll get to the SMBs in a second. I promise. But I think that this, you know, if, if an enterprise can benefit from this, I think an SMB can as well, because they have less internal change management. They can, you know, they have more ability to control their process. But what happened with Scotia Bank is in the past when they would hire a co-op student, which is basically a paid internship or in their internship roles, they used to only hire people with a finance and business background. So you think an SMB can only hire people that have that expertise, well, Scotia bank was like, we're only going to hire finance and business backgrounds. And they were only hiring from about five or six of the top universities when they introduced Plum and eliminated resumes, they started hiring from 33 different colleges and universities and 40% of their hires were actually STEM and arts backgrounds.
Cailtlin (13m 58s):
And the result of that is that they increased their hiring of underrepresented minorities to 60%. They ended up hiring just over 50% women and they had double the retention as a result of all of this. So all of these incredible outcomes were just, you know, they weren't the goal, they were just the outcome of screening in people that they hadn't looked at before. So if I think about SMBs, this is a massive opportunity to screen in talent that the rest of the market isn't looking at! You know, I first hire when I was running a tech company with somebody that had seven years of waitressing experience, you know, didn't know how to use Excel. My second hire was a high school student who was only experience was cutting grass.
Cailtlin (14m 39s):
The next person had dropped out of college to join the carnival. And they were all incredible top performers that would never have got an interview if it hadn't been for the assessment saying that they had this incredible alignment.
Chad (14m 50s):
What did that person do in the carnival?
Cailtlin (14m 52s):
I do not know, but I can tell you.
Chad (14m 55s):
Oh come on.
Cailtlin (14m 55s):
I can tell you that it was a woman who had never been part of the tech scene and now to this day, she's in customer success in the tech industry! It was the on-ramp. We have over, we have almost 60% of women who work at Plum and the majority of them had never been part of a software as a service tech company before, but now I have to work really hard to make sure they don't get poached by bigger, huge companies in town like Shopify, because there are incredible talent that now show up on resumes as fitting the pattern. But they love working at Plum because we saw something in them and they want to build a company that can help find more people like them that would not get through the initial screen with a resume.
Chad (15m 35s):
Gotcha.
Cailtlin (15m 35s):
Yeah.
Joel (15m 36s):
I'm going to go with sword swallower, Chad, my guess.
Chad (15m 39s):
Yeah. And back away from the mic Cheeseman, I'm going to give you an argument here. So Scotia Bank totally love it, interns for the most part. Right and then intern to hire. College recruiting really to be quite Frank is not great. Kind of sucks in to say that they are doing better, doesn't surprise me. I think I can actually take, you know, probably a carnival out and actually do better recruiting than most big Fortune 50 companies do in colleges. So that doesn't surprise me. What I would love to see is the opportunity for guys like Joel and I who've been in an industry for 20 years to not have to use a stupid resume. Is that something that you can see happening in the next few years?
Chad (16m 20s):
That's the big question.
Cailtlin (16m 22s):
That is a great question. So with the vast majority of our customers, they're actually using us on all levels across the organization. So Intact Insurance, we've been a customer for almost five years and they use us across all roles across North America. It's not just entry level. And it's being given to everybody right out of the gate.
Chad (16m 42s):
They don't use resumes either.
Cailtlin (16m 43s):
They use resumes after. So that's the key thing here is that I don't think we're going to get rid of resumes in the near term entirely, but what's happening is that if you get a hundred people that apply for the job and they were to go based on experience and you know, you may be number 80 out of a hundred for a particular role and they may not even get to you. I mean, it's scary how few resumes actually thoroughly get read and sometimes you get halfway through the pile and sorry, I'm not even going to look at it. So you may have been number 80 in the pile, never even looked at or that's how they stack ranked you against everybody else. Whereas with Plum, if you have the right talents, like let's say needs somebody that's really good at persuasion and really good at executing, then you may have a 95 match for that role.
Cailtlin (17m 32s):
You're in the top 5% of the entire population. They're taking a look at you and then they're deciding, Hey, do we think there's a transferable nature to your past experience? My biggest thing is, are they worth a phone screen? And I would hope that if that was a role that you were going to excel in, you would get the phone screen with Plum. And if it's not, and you're a 30 match per se, the reality is with the case of Scotia and our other customers, you're not just matched to one role. You're matched all the roles in the organization. So you're able to maybe be a 30 for one role, but there'll be like, actually he's a 92 for this other role. Hey Chad, would you be interested in having a conversation about this other role because you're incredibly aligned to it and you might be like, I never even thought of that title.
Cailtlin (18m 13s):
Yeah. I'd be interested in having a chat.
Joel (18m 15s):
Let's talk about employment branding for second. Do assessments have an image problem and do companies risk hurting their brand when they use assessments? And I'm going to give you a couple examples. I went out to Reddit to a recruiting hell conversation. I'm going to read a couple quotes on assessments and what job seekers thought about them? First one is quote "these assessments feel mostly pointless. I have a feeling that with these assessments, they just steal ideas or just make me do work that they don't want to do themselves. On one, I had to simply log loads of details into an Excel spreadsheet. It made me feel used. And like I wasted my time." Another one said, quote, "Time is money. No one needs to do any of these assessments.
Joel (18m 56s):
If employers can't gauge your skill level by reading your CV and putting you through two interviews, they have no business hiring you in the first place." How should companies think about assessments and their employment brand and how job seekers think about their company?
Cailtlin (19m 10s):
I think it goes back to what we were talking about before. There's a lot of really bad assessments out there. And you know, we're talking about 50 years of assessments like Myers-Briggs and DISC being used and that's really polluted the waters. What I would say is if you go onto LinkedIn and you look up Plum top talents, you'll see by the hour people talking about how grateful they are, that they got the chance to complete a Plum assessment. They're thanking the employer for asking them to do it because we've done what the industry has been asking for. Literally there was research that just came out by Lighthouse Research and Advisory Group, where they talked about how 68% of diverse candidates actually prefer an assessment or a video interview over a resume because of the dynamic nature and the single greatest thing that people want is results back.
Cailtlin (19m 56s):
They want to have value from the data. Even if you don't get the job, they should be able to use it, to help them with their career. So from a employer branding perspective, this is a gift. Every single person that applies you are now enabling them with a language to better articulate what makes them exceptional. And they're posting those top talents on LinkedIn and thanking employers and saying they have better insights as a result. I think in this day and age, the thing that has shifted with COVID is that finally, we understand that we need to understand the humans behind the work and individuals don't necessarily understand what makes them exceptional against the millions of other people they're competing with. And so Plum gives them insight into what makes them exceptional and where they would thrive in their career.
Chad (20m 41s):
Okay. So let's talk about the wonderful world of auditing and the EEOC and OFCCP. So HireVue thought they could just go ahead and slip by because they put on an explainability statement that points the finger directly at their clients, trying to absolve any blame from themselves. So what are your thoughts about this? Is this a client's problem? This is they're using your tech, it's on them, or is this truly a shared responsibility?
Cailtlin (21m 9s):
I love that Matt sharer earlier this month when he was on your podcast, talked about what really needs to happen is there's more accountability from the vendor and I couldn't agree more. This is really about the vendor and the company having a true partnership. Bias is not just at one step bias can be in the assessment, but it can also be in how the assessment gets used and the only way to really, you know, eliminate as much bias as possible is to have a partnership where both have control of the flow and the data, And it's only when you put these two together. Data that the assessment has data that the company has, that you can really address bias correctly. And it's really not about just the hot potato.
Cailtlin (21m 52s):
You can't pass the buck. You know, there's a real responsibility. Like I said, from how we design the assessment from the ground up following absolute best in class to mitigate as much bias as possible to use this, to screen people in. But then also working with the customer to make sure, Hey, why would you use a resume first when Plum's data is four times more accurate? At what point is it appropriate to bring in hard skills, past experience? At what point is it appropriate to bring in other information, we really work with our customers as a partnership. And we really why we put so much work also into, you know, the new audit bias. The bar is really low right now because it is a new requirement and we're taking every opportunity to go above and beyond and try to say, even if the bar was higher, how would we go our farthest to ensure that we are showing that we have removed as much bias as possible, and that this is a benefit to the hiring selection process?
Cailtlin (22m 48s):
Really we'd like to see the entire selection process, go through an audit because there's a lot of things that companies do wrong, like lack of structured interviews, you know, lack of relevant job related questions and we think there's a real opportunity to work with customers to improve that process. And we're doing our part and going above and beyond wherever we can.
Chad (23m 9s):
We're going to have to get you to a safe house right after this because after vendors hear you say that, they're going to say, holy shit, we're not ready for this. So, since you listened to the Matt share interview, which, you know, I thought it was definitely awesome. He actually pointed out that the validation reports that most organizations, most vendors are actually putting out are only 2% better than picking names out of a hat. The question is, why should I spend so much money on 2% better or are you saying validation in your system process is so much better than just 2%?
Cailtlin (23m 47s):
Yeah. This is four times greater at predicting on the job success than a resume. 2% is not something that you should be spending your money on. You can do so much better than that. And that's why it's important to have third parties be able to back up your claims. And that's what we're doing is not only just having one person do it, a bias audit for us, we're actually going to three independent people, three different experts in industrial organizational psychology and statistics. And we're looking at making sure that similar to how you validate science in general, you don't just take the claim of the company. You make sure that other experts in the field can back up, that it's designed the right way, that the results are accurate and that you are getting that increase in improvement that is being claimed.
Cailtlin (24m 30s):
And so we're really excited. We're actually going through that process right now and hope to have that publicly published by September.
Joel (24m 36s):
All right, Caitlin, here comes the hard ball. You ready?
Cailtlin (24m 40s):
Yep!
Chad (24m 40s):
Uh oh.
Joel (24m 40s):
Better Canadian entertainer Bieber or Bublé?
Cailtlin (24m 45s):
Oh that's tough.
Chad (24m 47s):
How is that tough?
Cailtlin (24m 48s):
I would go for Bublé, but I'm sure a lot would not agree with me just in terms of, Bieber's got a lot of supporters that grow in the younger generations. But, I mean, they're both great. Can I go with Ryan Reynolds?
Chad (25m 3s):
Yes, of course.
Joel (25m 4s):
All right. Everybody Caitlin McGregor. She's a little confused about Canadian entertainers, but she knows her assessments. Caitlin if someone listening wants to know more about you and your company and assessments, where would you send them?
Cailtlin (25m 19s):
plum.io? And you can actually take the Plum assessment right off of our website as well. Plum.io/ds for discovery survey. And you can see what your top talents are as well.
Chad (25m 33s):
Nice!
Joel (25m 33s):
Love it. Chad another one in the can!
Chad & Cheese (25m 35s):
We out
OUTRO (26m 28s):
Thank you for listening to, what's it called? The podcast with Chad, the Cheese. Brilliant. They talk about recruiting. They talk about technology, but most of all, they talk about nothing. Just a lot of Shout Outs of people, you don't even know and yet you're listening. It's incredible. And not one word about cheese, not one cheddar, blue, nacho, pepper jack, Swiss. So many cheeses and not one word. So weird. Any hoo be sure to subscribe today on iTunes, Spotify, Google play, or wherever you listen to your podcasts, that way you won't miss an episode. And while you're at it, visit www.chadcheese.com just don't expect to find any recipes for grilled cheese. Is so weird. We out.