👋 Welcome to CTO Teachings’s blog. While we are a recruiting company, we happen to provide a free blog for people just like you. It’s packed with hard-won lessons from CTO—who’s helped managers rise to Engineering Director and CTO roles.
So you want to hire an engineer that's good at AI? Whether it's ChatGPT, Claude, X, or whatever your favorite tool is, most companies are doing this completely wrong.
Speed Over Algorithms
First thing I like to do is hire for speed. As I mentioned in my previous post about interviewing, you need to baseline your interview process. Run it on people already on your team with the caveat of "look, I'm using this because I want more people like you." You know – people who are cranking out tickets, turning them over fast, and getting stuff done.
One big mistake I see everywhere? Creating these crazy algorithm interviews. In reality, we're not dealing with complex algorithms day-to-day. We're dealing with complex business problems – making something work for different roles, security considerations, real-world stuff.
Generally, I might need 1 out of 10 people who can really do advanced algorithms. And just by hiring 10 fast people, I naturally get 2 or 3 of those people who are really good at algorithms anyway.
Most of your engineering work involves understanding legacy code, debugging production issues, integrating third-party APIs, and dealing with edge cases that no algorithm textbook covers. When someone gets stuck on a tricky OAuth implementation or figuring out why a microservice is timing out intermittently, they're not reaching for their knowledge of binary tree traversals. They're firing up Claude or ChatGPT and iterating through solutions.
Let Them Use AI in the Interview
My suggestion for hiring engineers that are great at AI? Give them a problem. Let them use AI to solve it. Tell every person in the interview upfront that this is how it works.
Create a standard that you're not going to help them. Why? Because when people start working on the job, they get blocked, try to solve something, and sit there by themselves for an hour before they ask someone else for help.
I've seen too many interviews where candidates get a sanitized coding problem that has one clean solution. That's not how real development works. Give them something messy – maybe a bug in a React component that's not rendering properly, or ask them to add a feature to an existing codebase with incomplete documentation. Watch how they interact with AI tools to understand the problem, explore solutions, and validate their approach.
The best AI-assisted engineers don't just copy-paste code. They ask follow-up questions to the AI, test edge cases, and understand the trade-offs of different approaches. They know when to push back on AI suggestions that might work but create technical debt.
The 5-Minute Rule
To simulate this real working environment in an interview, sit there for 5 minutes. 5 minutes is a really good mark, and yes, you will feel awkward silence in those 5 minutes. They'll sometimes ask questions, and if you've set up the interview correctly, you'll have all the information there, but you can't answer any questions until the 5-minute mark. At that point, you're allowed to give another hint.
This slows down the interview and baselines it more accurately. You get a better comparison of what each person will really do on the job.
During those 5 minutes, pay attention to how they approach the problem. Do they immediately start coding, or do they spend time understanding the requirements? Are they prompting the AI with vague requests like "fix this code" or are they being specific about what they're trying to achieve? Good AI users know that garbage prompts produce garbage outputs.
I've noticed that strong candidates will often challenge AI suggestions. They'll get a solution from ChatGPT, read through it, and say something like "this looks like it might have a memory leak" or "this won't scale if we have thousands of users." Weak candidates just implement whatever the AI suggests without thinking.
Avoid the Helping Trap
What happens in a lot of interviews is people crack and help someone 60 seconds later. Meanwhile, they might see someone else who's making progress, and they won't help that person 60 seconds later. This means the person you helped earlier has a significant advantage, and your comparison is worthless.
The Reality Check
If you're going to hire for AI skills, I highly suggest you actually let candidates use AI in the interviews. A lot of companies still aren't doing this, which makes absolutely no sense.
What you're really looking for – engineers who don't just listen to the AI engine every time and program whatever it says. Because as we all know, AI can be wrong quite often, or point you down a solution that works today but isn't scalable. AI doesn't have that advanced thinking capability yet.
You want people who can use AI to boost their productivity while still thinking critically about what it's telling them.
Think about it this way: every engineer on your team will be using AI tools daily. The ones who succeed will be those who can leverage AI for rapid prototyping, debugging, and learning new frameworks, while still maintaining code quality and architectural thinking. The ones who struggle will be those who either refuse to use AI or blindly trust everything it outputs.
I've seen engineers cut their debugging time in half by using AI to explain error messages and suggest fixes. But I've also seen engineers waste entire afternoons implementing AI-suggested solutions that were fundamentally flawed because they never stopped to understand what the code actually did.
Share Your Experience
I'd love to hear your thoughts on this. What other things are people doing when interviewing to find engineers that are great at using AI? How are you separating the people who can actually leverage these tools effectively from those who just copy-paste whatever the AI spits out?
The goal isn't to find people who can replace AI – it's to find people who can work with AI to solve real business problems faster and better than anyone else.
If you are ever looking for a job or expanding your team, connect with Shelly, Yuliia, or Harold on LinkedIn.
Thanks for reading,
CTO Teachings