What an AI engineer actually does

It is the Monday after Diwali at Crumb & Co., and the support inbox has 1,400 unread emails.

Arjun scrolls through them while his chai goes cold. They are almost all the same six questions. Where is my hamper? Does the kaju katli contain gluten? Can I change the delivery address? Why was I charged twice? Can I get a refund, it arrived squashed? Do you deliver to Hinjewadi? Crumb’s three support staff answer about 120 emails a day between them, at roughly four minutes each. At that pace the backlog will take them almost two weeks, and new emails keep arriving.

Meera, Crumb’s founder, pulls up a chair. “Every company I know has an AI assistant now. Can we have one? Something that drafts these replies so the team only has to check and send.” She pauses. “And please tell me it will not invent our refund policy. Last year a temp promised a customer a full refund on a ₹4,000 hamper and we had to honour it.”

Arjun is a backend developer. He built Crumb’s order API and keeps the online shop running. He has used ChatGPT to write birthday messages. How hard can this be?

Arjun’s first attempt

That evening Arjun signs up for an API account with a model provider. An API (application programming interface) is a doorway that lets one program ask another program to do something; here, it lets his Python code send text to a powerful language model running in the provider’s data centre and get text back. He writes ten lines of Python:

import anthropic

client = anthropic.Anthropic()  # reads the secret API key from the environment

email = """Order CR-20931 never arrived. It was a birthday cake for my
mother and the party was yesterday. I want a full refund. - Kavya"""

response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=500,
    messages=[{"role": "user", "content": f"Reply to this customer email:\n\n{email}"}],
)
print(next(b.text for b in response.content if b.type == "text"))

You do not need to understand every line yet; you will run code like this yourself after the setup lab. What matters is what came back. It looked something like this:

Dear Kavya,

I'm so sorry your mother's birthday cake didn't arrive in time. That's
not the experience we want for you. I've processed a full refund to your
original payment method; you should see it within 5-7 business days.
As an apology, here's a 20% discount code for your next order: SORRY20.

Warm regards,
The Crumb Team

It is warm, fluent and well written. Arjun is delighted for about ten seconds. Then he reads it again.

The model has not processed any refund; it cannot touch Crumb’s payment system. Crumb’s actual policy, from page 47 of the store operations handbook, is a full refund only if the order is more than two hours late, otherwise store credit. The code SORRY20 does not exist. And it never checked whether order CR-20931 was actually late: according to the order database it was delivered yesterday at 4:10 p.m., signed for by “Ramesh, security”.

If Crumb’s team had sent this draft, they would have refunded a delivered cake, honoured a fake discount code, and taught a customer that complaining gets her free money. Meera’s nightmare, in eight lines.

What did the model need that it did not have?

Arjun shows Priya, the senior AI engineer who joined Crumb last month from a large tech company. She reads the reply and nods slowly.

“It is a good email,” she says. “For a bakery that does not exist. Let me ask you something. If we hired a brilliant new support person today, the best writer in Pune, and sat them in front of this email on their first morning with no training, what would they write?”

“Probably… something like this,” Arjun admits. “Polite. Made up.”

“Why made up? They are brilliant.”

“Because they do not know our refund policy. They cannot see the order database. Nobody told them not to promise things. And nobody is checking their work.”

Priya writes four words on the whiteboard: instructions, facts, tools, checking. “You just described the job,” she says. “The model is the brilliant new hire. It writes beautifully and knows a lot about the world. It knows nothing about Crumb. An AI engineer is the person who gives it a proper briefing, hands it the right files, lets it look things up in the right systems, and checks its work before a customer sees it.”

“So I do not need to build a smarter model?”

“You could not if you tried. Training a model like this takes a team of researchers, thousands of specialised chips and months of time. Somebody else did that. Think of a restaurant. The chef does not grow the wheat or raise the cows. The chef takes excellent ingredients and turns them into a meal people want, served hot, every night, without poisoning anyone. You are the chef. The model is the flour.”

Show the answer

1. The order record: was CR-20931 delivered, and when? (It was.)
2. The refund policy: full refund only if more than two hours late, otherwise store credit.
3. The house rules: never promise refunds or discount codes yourself; escalate anything over ₹2,000 to a manager.
4. The customer’s history: is this her first complaint or her tenth?

Every one of these lives in Crumb’s systems, not in the model. Getting the right information to the model at the right moment is a large share of an AI engineer’s job.

Following one email through a real system

Priya sketches what a properly built version would do with the same email. Follow Kavya’s message through it, step by step, with the real values at each stop.

flowchart LR
    E["Kavya's email"] --> A["Crumb app (normal code)"]
    A --> P["1. Instructions"]
    A --> R["2. Facts: handbook search"]
    A --> T["3. Tools: order lookup"]
    P --> M["Model via API"]
    R --> M
    T --> M
    M --> D["Draft reply"]
    D --> C["4. Checks"]
    C --> H["Human approves and sends"]

Step 1: Instructions. Before the email, the app sends a fixed set of standing instructions, called a system prompt:

You are Crumb & Co.'s support assistant. Write a draft reply for a human
to review. Use only the policy passages and order data provided. Never
promise refunds, credits or discount codes; say a team member will confirm.
If the order data contradicts the customer, say so gently and ask for details.

Step 2: Facts. The app searches the 200-page handbook for passages related to “late delivery refund” and finds two short ones, the refund policy and the “delivered but customer says not received” procedure. This idea, fetching relevant text and handing it to the model with the question, is called RAG (retrieval-augmented generation). You will build it properly in Module 09.

Step 3: Tools. The app gives the model a tool: a function it may ask the app to run. The model replies, in effect, “please run get_order('CR-20931')”. The app runs it against the real order database and returns:

{"order_id": "CR-20931", "status": "delivered",
 "delivered_at": "2026-11-08 16:10", "signed_by": "Ramesh, security",
 "promised_window": "15:00-17:00", "total_inr": 1450}

Letting the model request actions like this is called tool calling. A model that keeps calling tools in a loop until a task is done is called an agent.

Step 4: The draft. Now the model has instructions, policy and the order record, about 1,100 tokens of input in total (a token is a chunk of text, roughly three-quarters of an English word). It writes about 150 tokens back:

Hi Kavya, I'm so sorry for the worry on your mother's birthday. Our records
show order CR-20931 was delivered yesterday at 4:10 pm, within the promised
window, and signed for by "Ramesh, security" at your building's gate. Could
you check with your building's security desk? If the cake still can't be
found, reply here and a team member will look into it with you today.

Step 5: Checks. Before a human sees the draft, the app runs automatic checks. Does it promise a refund, credit or code? (No.) Does it mention the real delivery status? (Yes.) Is it under 120 words? (Yes.) Systematic tests like these, run across hundreds of real emails, are called evals (evaluations), and they are how you turn “it seemed fine” into a number.

The bill. With a mid-priced model, a call this size costs roughly half a rupee (about half a US cent; exact prices vary by model and change often). The whole call takes about two to three seconds. Drafting all 1,400 backlog emails would cost around ₹700, against roughly 90 hours of staff time to write them by hand. The humans still review every draft, but reviewing takes about 40 seconds instead of four minutes.

Notice how small the model’s part was. It is one box. Everything else was ordinary software: a database query, a search, some string checks, a review screen. That is the job.

What an AI engineer actually does all day

Now the pieces have names. Most AI engineering work falls into five areas, and you just saw all five in one email:

1. Prompting. Writing the instructions so the model behaves the way your product needs. Not magic words; clear, specific briefing. (Modules 02 and 03.)
2. Context and retrieval. Getting the right facts into the model’s input at the right time, from documents, databases and conversation history. (Modules 07 to 10.)
3. Tools and agents. Connecting the model to your code so it can look things up and take actions, safely. (Modules 06, 13 to 18.)
4. Evaluation. Measuring quality with real examples, so you know whether a change made things better or worse. (Module 23, and threaded throughout.)
5. Production engineering. Speed, cost, reliability, security, monitoring. Making it survive real customers at 11 p.m. on Diwali. (Modules 19 to 26.)

A typical week mixes all of them: an afternoon reading 50 conversations where the bot did badly, a morning rewriting instructions, a day wiring up a new tool, an hour arguing about whether a 2-second delay is acceptable.

Show the answer

Context and retrieval. The model is probably answering from an outdated handbook passage, or from nothing at all and guessing. The fix is to make sure the current store hours are in the documents the app searches (or available through a “store hours” tool), and then to add an eval question about Baner’s hours so the mistake cannot silently return. Retraining the model would be the wrong tool entirely: it is slow, expensive, and hours change every season.

How this differs from the other “AI jobs”

Meera has been listening. “So should I be hiring a data scientist? Or one of those machine learning people? My nephew says those are the real AI jobs.”

Priya draws a table. Job titles in this field are blurry and companies use them differently, but the main question each role owns is distinct:

Role	The question they own	A typical week
ML researcher	How do we make models fundamentally better?	Designing new architectures, running large training experiments, writing papers. Heavy maths.
Data scientist	What does our data tell us?	Analysing sales, running A/B tests, building forecasting models, presenting findings. Statistics.
ML engineer	How do we train and serve our own models reliably?	Training pipelines, data cleaning, GPU clusters, model serving.
AI engineer	How do we build a useful product on top of existing models?	Prompts, retrieval, tools, agents, evals, APIs, cost and latency work, shipping features.
Software engineer	How do we build reliable software?	Everything else. AI engineers are software engineers first.

The biggest difference is the starting point. A researcher starts from data and maths and ends with a model. An AI engineer starts from a user’s problem (1,400 unread emails) and ends with a product. The roles overlap, and a strong AI engineer borrows from all of them: a little statistics for evals, a little ML engineering when running open models.

“For your inbox,” Priya tells Meera, “you do not need a researcher. You need someone who can do what Arjun just watched me sketch. Which is what Arjun is going to learn.”

The phrase “AI engineer” became common around 2023, when powerful models became available through simple APIs. Before that, nearly every AI feature needed a team training its own model. Now one developer can build in a few weeks what used to need that whole team.

Where it breaks: the three honest truths

Before Arjun gets carried away, Priya writes three more lines on the board. These are the things that make AI engineering genuinely different from the backend work Arjun already knows, and they are why this course exists.

1. The output is not guaranteed. Call a normal function twice with the same input and you get the same answer. Send the same email to a model twice and you may get two different drafts, both reasonable, or one wrong. Arjun reran his naive code three times: the fake discount code was SORRY20, then CRUMBCARE, then absent. You design for variation instead of assuming it away.

2. Models can be confidently wrong. The refund email was not a crash or an error message. It was fluent, polite and false. A model producing believable but untrue content is called a hallucination. Instructions, retrieved facts and tools make hallucinations much rarer. Nothing makes them impossible, which is why Crumb’s design keeps a human approving every draft for now.

3. “Correct” is fuzzy. Is “Our records show it was delivered” a good reply? Only if the tone is kind, the facts are right and the policy is followed. Measuring that across thousands of emails is an engineering discipline of its own, and it is the skill most beginners skip.

A good AI engineer is the person in the room who says “it drafts a usable reply about 90% of the time, it never promises refunds in our 300-email test set, and here is how we catch the rest.” Not the person who promises magic.

What you will be able to build

This course is a six-month programme in two parts. Part 1 (months 1 to 5) is AI engineering: building products with models, exactly the work above. Part 2 (month 6) is deep learning foundations: opening the hood to understand how the models themselves are trained. Along the way you will build about 17 projects. Roughly:

When	You learn	Example projects
Month 1	Orientation, Python, how language models behave, your first chatbot, prompting	A command-line support bot for Crumb
Month 2	Real text tasks, structured apps, function calling, embeddings	A ticket-routing support agent, an embeddings explorer
Month 3	Vector databases, RAG, scaling RAG, Hugging Face models	A chatbot over YouTube transcripts, an enterprise RAG system
Month 4	Agents, LangGraph, memory, voice, MCP	A coding agent, a voice agent, an AI code reviewer
Month 5	Streaming, local models, caching, tracing, evals, security, deployment	A trading research agent, production hardening
Month 6	Maths, PyTorch, CNNs, sequence models, transformers, fine-tuning	A neural network, an autoencoder, a multi-input OCR model

The finale is a set of portfolio projects: a ChatGPT-style platform, chat with any document, an AI search engine, an AI app builder and an AI interviewer. The months are a guide, not a contract.

Try it yourself

Practise seeing AI products the way an engineer does. Pick an AI feature you have used: a support chatbot, a “summarise this” button, an email reply suggestion. On paper, sketch it using the four words from Priya’s whiteboard: instructions, facts, tools, checking. For each, guess what that product does.

Show an example answer

An airline support chatbot, for “my bag did not arrive”.

• Instructions: “You are a polite airline assistant. Never promise compensation amounts. Hand over to a human for anything about lost medication.”
• Facts: the lost-baggage policy page, retrieved because the question mentions a bag.
• Tools: look up the customer’s booking; open a baggage claim; check the bag’s last scan location.
• Checking: someone measures how often it quotes the right policy and how often it escalates correctly, probably on a few hundred past conversations.

If you could not fill one of the four, good: that is a question this course will answer.

Common mistakes

• Thinking you must train your own model first. Almost no product starts there. Start with an existing model through an API; fine-tune only when you have evidence you need to (Module 33 shows when).
• Thinking the model is the product. It is one ingredient. Users judge the whole meal: speed, accuracy, tone, and what happens when things go wrong.
• Treating a good demo as done. Arjun’s first draft looked great on one email. A system that works on five emails often fails on the five-hundredth. Evals turn “it seemed fine” into a number.
• Skipping the boring engineering. Logging, error handling, rate limits and security matter more in AI apps, because the model adds a new kind of unpredictability.

Recap

• An AI engineer builds products on top of models that others trained, usually reached through an API.
• The model is a brilliant new hire who knows nothing about your business: your job is instructions, facts, tools and checking.
• The work spans prompting, retrieval (RAG), tools and agents, evaluation, and production engineering.
• It differs from ML research (making better models) and data science (learning from data), though the roles overlap.
• Three honest truths: outputs vary, models can be confidently wrong, and quality must be measured.

On Friday, Arjun walks Meera through it. “The AI is like a very good writer on their first day,” he says. “It does not know our policies or our orders. So we give it the policy pages, let it look up the order, tell it never to promise refunds, and test it on hundreds of old emails before anyone trusts it. Your team still presses Send.”

Meera nods. “And the refund promise?”

“Can only happen if our checks miss it. And we will measure how often they do.”

“Good,” Meera says. Then, at the door: “One more thing. The newspaper says AI, machine learning, deep learning and generative AI as if they are the same. Are they?”

Arjun opens his mouth, and closes it. That is the next lesson.

Go deeper

• Building effective agents: a clear, practical essay on how real teams build with models. Read it now for the flavour, and again in Module 13.
• Anthropic documentation and OpenAI documentation: skim the home pages to see what an AI engineer’s toolbox looks like.