Tech interviewers can't tell when candidates use ChatGPT to cheat in coding rounds, experiment finds

• Thousands of tech workers are on the job hunt after brutal layoffs and changed industry dynamics.   
• An experiment found it's easy for people to cheat in tech job interviews with ChatGPT.
• It's also tough to get caught, which may lead to changes in how companies conduct tech interviews.

A historically tough job market for tech workers combined with the explosion of generative AI may result in a rise in tools like ChatGPT being used to breeze through early rounds of interviews.

As an industry-wide reckoning with ZIRP, or “zero interest rate phenomena,” continues, tech workers affected by mass layoffs are trying to get a break where they can. One possibility is the coding rounds of the typical tech job interview, where candidates are prompted by interviewers to solve technical problems. These interviews are usually conducted by a hiring manager for a team or company, and problems proposed are often taken from or based on the LeetCode interview prep platform. So, job candidates can easily use something like ChatGPT to effectively cheat their way to fast solutions, according to a recent experiment conducted by interviewing.io, a popular platform for mock tech interviews and industry recruiting.

Not only is it easy to use ChatGPT to ace a typical coding round of a tech interview, interviewing.io found in an experiment last month, based on 37 tech job interviews, that no interviewers could tell when a candidate had used ChatGPT to answer or solve proposed problems.

Aline Lerner, the founder of interviewing.io who previously worked as a tech recruiter and earlier an engineer, told Business Insider that people conducting tech interviews today have come to expect a type of “perfect answer” from job candidates. That leads to answers provided by something like ChatGPT, which is trained on information scraped from the internet, including massive amounts of technical coding problems and solutions from LeetCode, “to sound normal.” 

“Now, through no fault of their own really, an interviewer can be biased against the candidate that is actually solving the problem for the first time,” Lerner said, giving a less polished but more creative answer to a problem.

While no one was caught or even suspected of using ChatGPT during the interview process, interviewing.io did find that ChatGPT was much less likely to give correct answers to proposed interview problems the more unique they were. 

When coding problems were taken verbatim from LeetCode, job candidates using ChatGPT solved correctly 73% of the time, interviewing.io found. When LeetCode problems were modified slightly, answers were still correct 67% percent of the time. When a problem was custom, however, ChatGPT answers were correct only 25% of the time. That’s far worse than how tech job candidates fare on their own. Relying on their own abilities, interviewing.io found that job candidates passed coding rounds of interviews 53% of the time.

Despite a rise in major tech companies using mostly internal generative AI tools to help engineers write and fix code more quickly (Meta uses a tool called Metamate, once called Code Compose; Google just renamed its tool Goose; Microsoft uses CoPilot), jobseekers are still largely expected to do such work on their own during interviews. However, a rise in cheating by using generative AI is on the way, and Lener says the modern tech interview process, at least the way the coding rounds work, is set to break because of it.

“Up until a year ago, it was pretty hard to cheat on those. Now, you can literally just put it in ChatGPT and get an answer in real time." Lerner said. "And the tools are going to get better, it’ll get easier and easier… so there will be more cheaters, and we will hit a breaking point, and companies will have to do the work of making a new process.”

Are you a tech employee or someone with a tip or insight to share? Contact Kali Hays at khays@insider.com or on secure messaging app Signal at 949-280-0267. Reach out using a non-work device.