The Wrong AI Tool Is As Bad As the Wrong Hire

Table of Contents

If you are like me, one of your biggest nightmares is hiring the wrong engineer.

That engineer who won’t play well with the rest of the team, gets into technical fights with the most senior members, and works on side projects he/she believe are more important for the company.

I don’t like micromanagement; it is too time-consuming. I also don’t want to rush features that will harm my team’s reputation.

When I build a team, I make sure I can trust each engineer. That I don’t need to babysit them, and they will take 100% ownership of what they are creating.

We’ve got a new nightmare to face: AI tools can cause the same harm as the wrong hire.

Do you trust your LLM model?

The first LLM I tried was GPT-3.5. It was pretty bad at content creation (articles) but somewhat decent for coding. Then GPT-4 came out, and it was a game changer.

I was hyped by the dream of “AI can do all the coding for me,” so I pushed it to the limit: I copied and pasted large source files into ChatGPT to make sure it had enough context to produce code very similar to my style. It was tedious and frustrating (especially with front-end code).

Then GitHub Copilot and Cursor came out. The breakthrough was that they were integrated natively with Visual Studio, so there was no need to copy and paste anymore. I could even change the model and use GPT-4 or Claude.

At first, it did not work out for me. It was convenient for prompting, but the code quality wasn’t there.

Then GPT-4o came out, and then the o3-mini model — GitHub Copilot created this inline editing, plus the Copilot Edits and Chat (ask any question about the codebase). It reached the point where it was actually useful.

Can an LLM replace a bad engineer?

Yes. You can even use the right LLM model to evaluate whether an engineer will ruin your culture – but that’s a different topic.

I get why many leaders are freezing hiring and are very suspicious about implementing AI tools.

Let’s be clear here: if you incorporate the wrong AI tool (the wrong LLM model), it is far worse than hiring the wrong engineer. It will pollute your entire engineering team.

There are 3 factors in play here to consider:

Choose the right LLM model for the task: for example, o3-mini for code generation is great.
The tool should integrate natively into your workflow: If the AI tool doesn’t work well with the rest of your stack, you are better off using ChatGPT or any other Chatbot.
Write a prompts playbook: even if you have a tool with a pretty UI on top of predefined prompts (or prompt generators), you still need a personalized set of prompts that you know work for your particular case.

My Current AI Toolset

My AI toolset is pretty simple. I’ve been using it for almost three years, and I feel confident that it is augmenting my quality and speed.

LLM Models

I use o3-mini for coding; sometimes, GPT-4o is better for simpler tasks (like QA automation). If I need to work on something more complex, like writing down requirements and the first prototype implementation, I use GPT-4.5 or the o1 reasoning models.

Code Generation

I’m currently using GitHub Copilot, which integrates pretty well with Visual Studio.

I use the inline chat to make small tweaks to code blocks – the Copilot Edits for pretty much everything, and from time to time, the Copilot Chat to get answers about the codebase.

Chatbots

You will undoubtedly need the flexibility and freedom of interacting with an LLM via a Chatbot, especially with great features like Deep Research in ChatGPT.

This is where your prompts playbook will come in handy.

Custom-made tools

I’ve written many small tools that I use every day. My AI Toolset is 80% custom tools.

Make sure to allocate a budget to allow your team to write your prompts playbook and turn it into custom tools. The productivity gains will be amazing.

Other tools

I’ve tried Notion, but I thought the onboarding was too complicated (I already have JIRA/Trello set up).

I’ve also tried Sourcegraph Cody. It is faster and more accurate than GitHub Copilot in answering questions about your codebase.

Are your engineers AI-equipped?

My most trusted engineers work with me through my company, InTheValley. We are all AI-ready to help startups like yours.

Check out InTheValley services!

Author
Recent Posts

Leo Celis

Founder & CEO at InTheValley

I help founders like you fix engineering teams that should be moving faster. If you're scaling a startup, you've probably felt the pain: great people on paper, but execution feels slow. I've been building remote teams for Bay Area startups since 2005 — engineers you can trust who actually deliver and know how to use AI to ship faster, not slower.