Welcome to Death Match, North America, 2019, part one of four. This Chad and Cheese Death Match episode features Brian Rowe, CEO of Pez.AI. Death Match took place at TAtech on September 26th, 2019, in sunny, beautiful Austin, Texas, with a room full of TAtech practitioners. The bar was open, as usual, and Chad and Cheese snark was flying. Not to mention the judges were pitbulls. Enjoy after a word from our sponsor.
PODCAST TRANSCRIPTION sponsored by:Disability Solutions helps support and educate your workforce through disability awareness and inclusion training.
Chad: Death Match is brought to you by Alexander Mann Solutions. Hiring great people is no easy feat. There are new obstacles around every corner and the competition for talent is intense. We need to be bold in our approach to tackling these challenges. Alexander Mann Solutions is that bold next step. With 4,700 people around the globe delivering market, leading, recruitment outsourcing, and talent consulting services.
Chad: In early 2020, AMS will unveil an exciting new digital solution that will disrupt how you connect with job seekers and hire the best fit candidates. Now is the time to create purpose built solutions, focused on solving the unique challenges employers face when it comes to engaging and hiring people their business depend on. To learn more about how Alexander Mann Solutions is working with talent acquisition professionals around the globe, visit alexandermannsolutions.com.
Announcer: Hide your kids, lock the doors. You're listening to HR's most dangerous podcast. Chad Sowash and Joel Cheesman are here to punch the recruiting industry right where it hurts. Complete with breaking news, brash opinion and loads of snark, buckle up, boys and girls. It's time for the Chad & Cheese Podcast.
Peter: All right. So if you've not been to a TAtech event before, you are in for a memorable treat. You're not here to see me. So without further ado, let me introduce podcasters extraordinaire, Chad & Cheese.
Chad: Raise that drink, everybody. So thanks first and foremost, thanks Peter and TAtech for making sure that we had alcohol in the morning because a lot of us got in late. We need that. Yeah, keep the party going. So Death Match, who's seen a Death Match before? This is our third Death Match. There we go. So you know, that's why you're here, right? Death Match really a variation of Firing Squad. This is more the onstage live version. So obviously for listeners in the podcast, check out Firing Squad, every single one of these pitches today. Without further ado, today we have four startups. They will have two minutes to pitch. After those two minutes, we have a stunning panel of judges who includes Cindy Songne from Talroo, Quincy Valencia from Alexander Mann Solutions. Woohoo, give it up. And Robert Ruff, Sovren Technologies, everybody. All amazing sponsors [crosstalk 00:03:38].
Chad: Right out of the gate, first up on stage, we have a chatbot, Brian Rowe from Pez.AI. Bring it. Just throw them. They don't even need to know, just to hit him in the back of the head. There it is. Do you have your timer ready? All right. Now with pop sockets, I mean pop sockets aren't going to do it. Yeah, it's just pop sockets. There we go.
Chad: Ready to go?
Brian: I'm ready. All right. Time. Nobody has enough time these days. Employers say they don't have enough time to properly evaluate candidates. They also don't have enough time to respond to candidates. Why is that when all of these tools that we have are supposed to save us a lot of time? So that used to be my problem too. I didn't have enough time. Our hiring process was broken, very slow, and it changed all that by creating chatbots to interview every single candidate that we have. And that's allowed us to drop our hiring process, the time to hire from five weeks down to one week. It's also given us higher quality data to make decisions because we have these full interview transcripts as opposed to resumes which are completely biased. So in order to get the best transcripts possible, our chatbots can detect vague and generic responses. And when it does, it probes for more details just like a human would.
Brian: On the engagement side, our chatbots can answer questions that candidates have about the company or role. When it doesn't have the answer, it'll ask a human for help. And if they don't know the answer, they can tell the expert or chatbot to try somebody else. And so expert will eventually find that answer and will remember the answer for the future, like you'd expect. But it also remembers, too, the expertise of the person providing the answers so that they can ask that person for help in the future. Just like a human would. That's us. We're at Pez.AI. We create chatbots to streamline human interaction.
Joel: All right, I'm going to help you out real quick because you have time left. Where can we find out more about you?
Brian: Pez.AI.
Joel: Thank you.
Cindy: So how is your bot different from all the other bots out there?
Brian: So the key is that our bots provide intelligence around the way it interacts, right? So a lot of bots are scripted. Our bots can actually detect crappy answers and it'll actually challenge people to provide better answers. So I've taught machine learning, natural language processing for six years at the graduate level. So I've built a lot of custom models for our company, which allow us to differentiate in ways that other companies can't.
Cindy: Who from your team has talent acquisition experience?
Brian: Nobody.
Cindy: It's obvious on your site. I'll talk to you about that later though. On the expert and the human. I'm sorry. I know....
Joel: Into the mic, Cindy.
Cindy: No, one of the ...
Brian: Keep it coming, come on.
Cindy: What if the human they're asking the questions of is flawed?
Brian: Is, sorry?
Cindy: Does not have the right answers, is not a good hire for you.
Brian: I'm not following the question.
Cindy: You said the bot, if the bot doesn't have an answer, it asks a human for intervention.
Brian: Yeah. So for candidate engagement, that's right. So what's the question?
Cindy: With that human, we don't keep hires forever. What if you at a later point decide that human is not the right human, doesn't have the right answer. You get rid of that person.
Brian: Oh, I see. Right.
Cindy: But you still have that data.
Brian: So, the bot can, you can tell the bot that that person no longer works there or no, it's recorded by the bot. So the bot dynamically learns all the interactions and so the answer that the human provides, it'll remember that for the future. So then you can change that if you want through the authoring platform in the backend.
Quincy: Good morning. So first question. Pez.AI has two modules. Correct?
Brian: Right now, yeah. Sure.
Quincy: So you've got just interview me in the expert FAQ, correct?
Brian: Yeah, that's right.
Quincy: Do they interact at all? Meaning if I'm a candidate and I'm going through the just interview me process and I'm in the middle of answering your questions, but I have another question. Wait a minute. What, you know, whether it's about benefits or whether it's about the industry or it's something beyond just what the interview questions are, is the bot programmed and is the logic built in to be able to answer a question that's not when in the interview work stream and go back to the interview question?
Brian: Yeah. So you can actually stack the modules and any question, like if a candidate has a question that's not related to the interview questions, the bot will answer that and then continue back. It'll repeat the question so that they remember what the question was and it will continue on from there.
Quincy: Okay, got it. And where do you get your training data from?
Brian: So it's a little bit of a long answer. The, again, because I've taught natural language processing, machine learning, I actually built a custom models that is re-purposing a clustering model to do classifications. So I actually don't need as much training data as other systems.
Robert: Okay. To follow up on that though, what data are you training against? Where'd you get that data?
Brian: Well, it's all user derived, right? So for our expert bot, it's actually, it doesn't have any data to begin with and so it learns all dynamically. So there are some, we use some data sets for ... Like we create our own corpora for topic associations and things like that, but that's easily scrapable from the web. So we just generate that and then we build our models based on that.
Chad: Okay. So if I'm a human and I give an answer, your application, your bot learns that and then can give that answer again in the future. Is that right?
Brian: That's right.
Chad: How do we know if that answer is good?
Brian: Well, there are two ways, right? I mean we're kind of assuming that the person who is providing the answer is acting on good faith, right? But obviously answers can go stale. They can also be inaccurate. So in those situations, the person who's receiving the answer or who's asking the question can actually just say no, that's not right. And then it'll repeat the process of trying to find a better answer.
Joel: In addition to having a lot of competition, you have a lot of well funded competition. What's the plan in terms of sort of competing with, you know, the folks that have gotten 40, 50, $60 million or do you not need that much money to have a successful chatbot in recruitment?
Brian: Yeah. I don't think that you need that much funding to be honest. I think like from the discussions that I've had our technology speaks for itself. And so I think the way that we're solving problems are also a little different from the status quo. And. so people who appreciate that sort of innovation in the way that hiring is done are going to gravitate towards us.
Quincy: In the just interview module, can you talk to me a little bit more about that? Are the questions coming from actual companies? Is it just a series of questions that are appropriate for whatever particular job or skillset it is that you're asking them about? And then where do those answers go? This is actually a two part question.
Brian: Yeah, no problem. So the way the site works is that any company can create their own custom interview questions. We provide our own kind of cannon of questions that are based per role. And we actually use them ourselves for our own hiring. We primarily focus on behavioral and experiential questions. And so we ask people about, you know, how they've handled certain situations in the past. And then for the technical questions we also get into, you know, some programming things, design questions, things like that. And those are the types of questions that's easy for people to either pick and choose some of those or create their own from scratch.
Quincy: Okay, great. And what across the platform, so everyone who's using it, yourself internally and others, what is the completion rate for the interview and what's the conversion rate of those that are being interviewed to actual hire?
Brian: A good question. I think we've had about 1,500 people take the interviews, the completion rate, I am not sure. We haven't calculated those statistics yet. So I'm not going to make anything up. I'll just tell you. I don't know.
Robert: Appreciate that. So what's your current sales and partnership strategy?
Brian: So our sales strategy is basically as I think one of the judges has mentioned, we're not really an HR tech company. We're a AI tech company. And so we're not trying to create a brand in talent acquisition or HR. Really what we're looking for to partner with job boards, ATSs, other companies that are really deep in the space and then, you know, be a white label service provider.
Brian: So we've got partnerships with two large HR consulting firms and that strategy has worked well for us. You know, so we've got large enterprises using our software. They have no idea who we are, but we're powering, you know, a lot of HR questions behind the scenes. That's great for us because we're focused on creating the best possible chatbot experiences and intelligence in those experiences as opposed to trying to learn, you know, all the ins and outs of this industry.
Cindy: Is there a good time length that the interview should take?
Brian: Yeah. So, having a lot of people take the interview is one of the things that we discovered is that on average the interviews right now take about 45 minutes and there's kind of a good and a bad. So, the bad part is is that it hurts the completion rate. On the flip side, what we've seen is that people who actually go through the process of finishing it are much more interested in getting the job. And so it's a much greater signal for determining who do you actually, you know, who is actually interested in finding work and who's just kind of, you know, fishing. So...
Cindy: Sure. And you don't have the conversion rates. Do you have the completion rate? Do you know what your fall off rates are? And if someone does fall off, can they come back and pick up where they left?
Brian: Yeah, so we're building that in. We're basically, right now we have a sales team that's following up to get them to complete. I'd say, I mean, if I were to guess, I'd say it's probably about 65% but we've been going to job fairs where people are very motivated to, you know, to find work. So it's, you know, I'm not going to lie, the selection bias is very favorable to us, right? Because they're very motivated to get a job. So I think for passive people, they're probably, the completion rate's not going to be that high. But one of the things that we're working on is changing the way that the interview questions are set up. So it's more of a kind of a choose your own adventure where each interview module is like five minutes and then people can kind of pick and choose and construct, you know, an interview profile based on the questions that they're interested in answering.
Joel: So you've mentioned that you don't have a core competency in HR recruitment and your bot has a lot of other services or categories that it services, so customer service and other other industries. Whereas your competition in many cases does have, you know, sort of a core competency in recruitment and history in that industry. Do you think that your sort of lack of knowledge of the space will be an advantage or disadvantage and why? And if it's a disadvantage, how do you plan on overcoming it?
Brian: Yeah, I mean I'm not going to like, I've been... I think this is my third conference in HR tech this year and I've learned a lot from each one because I don't have a lot of information about the jargon, the vernacular. That said, I think hiring problems are pretty universal. I've hired, you know, gazillions of people in my career. I've looked for jobs. So to me I think it's an advantage because I do come with a fresh perspective. It may be a highly technical perspective, but it is, you know, it's a different perspective.
Brian: And I think, you know, in this time and age there's so much emphasis on data driven approaches to you know, making companies more efficient that what we're doing is essentially using chatbots to drive a lot of that efficiency. And so if that can be used within, you know, the HR space, then why not? And I think there are enough innovative people in the space that are willing to, you know, are interested in exploring that, which is why the partner strategy makes a lot of sense.
Chad: So in your choose your own adventure kind of land, the way that you're looking to modularize this, can that data be stored in more of a, just for like GDPR purposes and be able to be more candidate centric so that they can have candidate own profiles so that they don't have to go through a 45 minute conversation every single time they go to another company? They can own their own data?
Brian: Yeah. So that was actually the original idea for Just Interview Me is to build it as something that's like the common app for college applications because it is really tedious to have to answer those same questions over and over again. So the idea is as a job board, the candidates could basically, you know, decide whether or not they wanted their transcript public and then they could come up in search results. And then that way it, you know, it just cuts out so much of the process because you know, an employer is just already looking at those, you know, the answers. And so they don't have to do all the coordination to, you know, take an interview, fill out these application forms and whatnot.
Chad: So with the applicant tracking systems, and you said that's part of your partnership strategy, is that one of the types of strategies that you're trying to pull together right now? Or is that down the road? Is that a visionary type of a thing? Are you trying to do this GDPR candidate own profile with that data?
Brian: Yeah, that's something that we're doing right now. We actually built our own lightweight ATS in our platform that pulls in that data shows a... Essentially it shows each interview transcript anonymously. So the part around the removing the bias is you don't have to look at the resumes and so you don't know anything about their name, their gender or their age, whether or not they went to, you know, an elite university or not. And so it's really this anonymous apples to apples comparison. It's really cool. We realize though that we don't want to build our own ATS. And so it makes a lot more sense to partner with ATS vendors and then just provide a export integration mechanism so that it can be easily integrated into their system. And I think that we can, you know, that they're, you know, this is where we have to find an innovative partner because there is a little bit of a change to the workflow. People that we've talked to, they've asked how they can kind of, you know, shoehorn our technology into their existing process.
Brian: But I really think the value is, you know, throwing everything on the table and saying, "How can we make this process even more efficient using these new tools and a new way of doing things," right? Because I think that there's over-reliance on resumes and they're just, to me, I've never really had much success with them. And I think, you know, the data these days, there have been a number of studies showing how resumes really don't have any predictive power in terms of candidate success.
Brian: What does that mean? I can't read facial expressions.
Robert: I'm sorry, my business has a resume. So it was just like... If it takes 45 minutes for a human to, excuse me, for a bot to do the chat, how does a human look at that? Who reviews that and how long does it take that human to review the results of that? Because the bot's not going to hire them.
Brian: Yeah. So the review and evaluation, the way that we have it set up is that each person on the hiring team can review the transcripts, right? So it's anonymous and honestly it usually takes less than five minutes to review a transcript because a few things pop out. And through this process, these are the things that we're starting to build into the chatbot. One is that some people just provide a series of very terse, you know, two to three word responses. Not very great. And as a human, if I were, you know, actually conducting interview, I would ask them for more detail. So that's the genesis of that model.
Chad: Thank you very much. Brian Rowe, everybody.
Ema: Hi, I'm Ema. Thanks for listening to my dad, the Chad and his buddy Cheese. This has been the Chad & Cheese Podcast. Be sure to subscribe on iTunes, Google Play, or wherever you get your podcasts so you don't miss a single show. Be sure to check out our sponsors because their money goes to my college fund. For more visit chadcheese.com.