All roadmaps

AI Engineer

Ship LLM features, not just prompts.

The complete AI engineer path for 2026: what the job actually is, engineering foundations, choosing and calling models, prompts you can test, RAG, evals, agents and MCP, security, multimodal, open models and production - every topic explained, with free resources to go deeper. Fresher and experienced tracks.

Your progress0/163 topics · 0%

Progress is saved in this browser.

14 stages163 topics₹0 to follow

Filled nodes are the focus for this track; outlined ones can wait. Tap a topic to open it and mark it done, learning or skipped.

AI EngineerRelated roadmapsPython roadmapPrompt Engineering roadmapAI Agents roadmapAI & Data Scientist roadmapBackend roadmapSystem Design roadmapProve it as you goReading is half of it. This roadmap isbacked by a free, scored AI Engineercertification, a job simulation on a realtriage system and daily scenariochallenges - evidence a recruiter cancheck, not a claim.Get certified - freeJob simulationDaily challengesKnow the job before the toolsThe AI engineer job in 2026Which skills to learnPython, APIs &SQLCore vocabularyAI-assisted codingWhich role to prepare forAI eng vs MLeng vs DSFresher vsswitcherSectors, cities & payWhat companies actuallyexpectHiring testsnowDay-to-day workProjects & proofWhat gets you hiredWhat you'llbuildThe 90-day planYour proof stackEngineering foundations firstPython for AI engineeringPython fundamentalsEnvironments & dependenciesType hints & Pydantic modelsAsync I/O & concurrencyTesting with pytestErrors, logging & debuggingSoftware engineering basicsGit & pull requestsHTTP, REST & JSONBuild a small API serviceSQL & the data you queryConfig, env vars & secretsAI coding assistants, usedwellUnderstand the models you buildonHow LLMs work, practicallyNext-token predictionTokens & tokenisationContext windows & cut-offdatesSampling: temperature &top-pTransformer & attentionintuitionReasoning models & effortHallucination & failuremodesChoosing models & inferenceThe model landscapeQuality, latency & costtrade-offsReading benchmarks with careHosted APIs vs self-hostingModel versions &deprecationsCall models like a backendengineerModel APIsRequest anatomy: messages &rolesSDKs, raw HTTP & compatibleAPIsStructured outputs & JSONSchemaStreaming responsesTool (function) callingTimeouts, retries & backoffUsage, rate limits & budgetsPlaygrounds & consolesPrompts are code: version andtest themPrompt engineeringPrompt structure that holdsupSystem prompts & instructionpriorityFew-shot & step-by-steppromptingTemplates: data apart frominstructionsPrompt versioning &iterationContext engineeringChoosing what goes incontextLong inputs: position &noiseSummaries & compactionPrompt caching & stableprefixesGround answers in your own dataEmbeddings & searchWhat embeddings areChoosing an embedding modelSimilarity search & top-kVector stores & indexesHybrid search: keywords +vectorsBeyond RAG: classify,cluster, dedupeRetrieval-augmented generationThe RAG pipeline end to endParsing PDFs, HTML & tablesChunking strategiesMetadata, filters &freshnessRerankingQuery rewriting & routingCitations & grounded answersPermission-aware retrievalFrameworks: LlamaIndex,LangChain & coMeasure quality before you shipEvaluation & testingSuccess criteria & what tomeasureBuilding an eval datasetCode-based checks &assertionsModel-graded evals & theirlimitsRetrieval metrics: hit rate,recall, MRRTask success & end-to-endevalsRegression tests in CIComparing prompts & modelsfairlyError analysis & humanreviewEval tooling: promptfoo,DeepEval, RagasWorkflows first, agents whenneededAgents & workflowsDeterministic workflows vsagent loopsWorkflow patterns: chain,route, paralleliseThe agent loop: plan, act,observeState & memoryCheckpoints & resumable runsIdempotent actions & saferetriesStep, time & cost limitsHuman approval for riskyactionsAgent frameworks: pick oneMulti-agent systemsConnect models to tools anddataMCP & tool integrationsHosts, clients & serversTools, resources & promptsTransports: stdio &Streamable HTTPAuthorization for remoteserversBuild an MCP serverConnect clients & debug withInspectorMCP security: poisoning &scopesDesigning tools models usewellTreat model input and output asuntrustedAI security & privacyPrompt injection: direct &indirectHandling untrusted modeloutputLeast privilege for tools &agentsSecrets & credentialsPrivacy, PII & dataretentionTenant isolation & accesscontrolGuardrails & moderationRed-teaming your own appBias, fairness & harmBeyond text: images, documentsand voiceMultimodal & voiceImage & documentunderstandingSpeech-to-text &text-to-speechRealtime voice agentsMultimodal pipelinesImage generation &provenanceRun and adapt models youcontrolOpen-weight modelsOpen-weight vs closed modelsModel licences & usage termsHugging Face Hub & modelcardsRunning models withTransformersLocal inference: Ollama &llama.cppQuantisation & memory mathsServing: vLLM & compatibleAPIsFine-tuning & adaptationPrompting, RAG orfine-tuning?Supervised fine-tuning &LoRAPreparing training dataEvaluating a tuned modelPreference tuning: DPO &RLHFPretraining & distributedtrainingRun it in productionDeployment, reliability & costDeploying an AI serviceTracing & observabilityCaching responses safelyQueues, concurrency & ratelimitsLatency budgetsCost: routing, batching &budgetsFallbacks & gracefuldegradationCanaries, shadow tests &rollbackMonitoring quality inproductionAI gateways & model routersProve it with working softwareProjects & career evidenceCapstone: one feature, endto endPortfolio projects that showjudgementDesign docs & eval reportsContributing to open sourceHackathons, used wellInterviews: LLM systemdesignProve it: certification &simulationProjects that show judgementA support answerer that refuses to guessRetrieval over a small policy set with citations, a 'noanswer' path, permission filters and an evaluation set thatproves unsupported answers went down.A structured extraction serviceMessy documents in, schema-valid JSON out: validation, abounded repair loop, cost per document and a report of whatstill fails.A bounded tool-using workflowTwo or three tools behind an MCP server, step and cost limits,idempotent writes, human approval for the risky step and atrace you can replay.AI Engineer job simulationWhat most learners get wrongLearning three agent frameworks before writing one plain APIcall.Judging a prompt on the five examples it was written against.Shipping without an evaluation set, so every change is aguess.Trusting retrieved documents, tool results and model output asif they were instructions.Ignoring cost and latency until the first bill or the firsttimeout.Letting an agent loop run without step, time or spendinglimits.Fine-tuning to add knowledge that retrieval would keepcurrent.A portfolio of demos with no tests, no failure handling and nowrite-up.Credentials: what is free and what is paidOneRoadmap AI Engineer certification - freeScored deterministically and added to your Talent Graph, so arecruiter can verify it in one click.Free courses, paid certificatesMost courses on this map are free to read or watch. Someplatforms charge for a certificate of completion, and runningexamples can need a paid API key - each resource says which.Cloud vendor certificationsPaid exams from cloud providers can help where a job namesthat cloud. They are optional; working projects and evaluationevidence matter more.Get certified - freeAI Prompt Engineering certificationKeep goingGo deeper on the language and the systems this roadmap leanson, or specialise in agents once the core is in place.Python roadmapSystem Design roadmapAI Agents roadmapBackend roadmap

Everything on this roadmap

Every stage and topic on the map, in reading order. Click any topic to open it.

The AI Engineer roadmap, topic by topic

Every topic on the map in reading order, with what to learn and the resources we recommend. Main topics are open; open any subtopic to read it. Select a topic name to open it in the topic panel, where you can track your progress.

  1. 1. Know the job before the tools

    The AI engineer job in 2026

    No prerequisites - a good place to start.

    An AI engineer builds software on top of models someone else trained: a support assistant, a document extractor, a search box that answers in sentences, an agent that files tickets. The model is one component. The job is the system around it - what the model sees, which tools it may call, how its output is checked, what each request costs and what happens when the answer is wrong. This section answers the three questions every learner asks, each with the report behind it. Konsi skills seekhni hain? - which skills to learn, in what order, and the one most people skip. Kis role ke liye prepare karna hai? - AI engineer vs ML engineer vs data scientist, the fresher and switcher paths, and where the jobs are. Companies actually expect kya karti hain? - what hiring teams test, what the day-to-day really is, and the projects and proof that get an interview.

    Why it matters: The 2026 picture in five numbers. AI engineer is the fastest-growing job in the US for the second year running, with LangChain, retrieval-augmented generation and PyTorch as its top skills (LinkedIn, Jobs on the Rise 2026). 71% of leaders would rather hire a less experienced candidate with AI skills than a more experienced one without them (Microsoft and LinkedIn, Work Trend Index). More than 1.1 million public repositories now call an LLM SDK, up 178% in a year (GitHub, Octoverse 2025). 84% of developers use or plan to use AI tools, yet 46% distrust their accuracy (Stack Overflow, 2025 Developer Survey). Generative-AI job postings in the US rose from 16,000 to over 66,000 in one year (Stanford AI Index 2025, Lightcast data).

    Example: An insurer's AI engineer connects an existing model to approved policy documents, forces every answer into a fixed JSON shape, blocks it from changing a claim, tests it on 300 real customer questions and tracks the cost per conversation. That is the whole job in one feature.

    Watch out: Treating the role as prompt-writing. Check: if a project write-up mentions no tests, no failure handling and no cost, it describes a demo rather than engineering.

    You can now: describe the role in a paragraph; name the three questions this section answers and where each is answered; say which of the five numbers changes what you learn first.

    Fresher: read all twelve sub-nodes before the foundations stage. Experienced: skim AI-assisted coding, Hiring tests now and Projects & proof - they are what changed since you learned to ship software.

    Which skills to learn

    2026

    Learn first: The AI engineer job in 2026

    Engineering first, then the model, then measurement. The foundations are Python with types and tests, HTTP and JSON, SQL and Git, because every LLM feature is ordinary software with a model call inside. The model layer is prompts kept as versioned code, structured output, retrieval, tool calling, evaluation sets, and cost and latency per request. The habit that ties them together is using AI coding tools well: drafting fast and verifying faster. Companies hire for the foundations, promote for the measurement, and in 2026 screen for the habit.

    Why it matters: LinkedIn names LangChain, retrieval-augmented generation and PyTorch as the top skills of its fastest-growing job (Jobs on the Rise 2026), and its Skills on the Rise 2026 list puts prompt engineering, large language models and AI business strategy in the rising AI stack. Stack Overflow's 2025 survey of 49,000 developers found 84% using or planning to use AI tools while 46% distrust the output and 45% lose time debugging AI-written code - the verifying skill is the scarce one.

    Example: A fresher who can write a typed Python function with a test, call a JSON API with error handling, build one retrieval feature with a 50-question evaluation set and quote its cost per query is interview-ready for applied AI roles without ever training a model.

    Watch out: Starting with anything that has LLM in the name before you can write and test a plain function. Check: list the three sub-nodes here and say, for each, the level you have actually reached.

    You can now: name the skills in the order to learn them; explain why evaluation and cost count as skills; say which habit hiring teams watch in week one.

    Fresher: foundations before anything with LLM in the name. Experienced: your gap is almost always evaluation and cost, not another framework.

    Python, APIs & SQL first

    Learn first: Which skills to learn

    The three skills an applied AI job assumes before any model is mentioned. Python to the level of typed, tested functions: type hints, Pydantic models for data shapes, pytest, virtual environments, async basics. APIs to the level of a small service of your own: HTTP verbs, JSON, status codes, retries and timeouts, a FastAPI service with two endpoints that calls a model and returns a fixed schema. SQL to the level of the data you will put in a prompt: joins, aggregation and filters, enough to know what a model is being shown and what it is not.

    Why it matters: Python added about 850,000 contributors on GitHub in a year, the largest growth of any language, because it is the language AI work is done in (GitHub, Octoverse 2025). The same report counts more than 1.1 million public repositories calling an LLM SDK, almost all of them ordinary services with a model call inside - which is why the ordinary parts are tested first.

    Example: Extracting the invoice number, date and total from a PDF's text is a typed Pydantic model, one model call with a JSON schema, a retry on malformed output and a test on twenty real invoices. No framework is needed.

    Watch out: Jumping to agent frameworks before you can write and test a plain function. Check: before the model stages you can write a typed function, test it with pytest, and call a JSON API with error handling.

    You can now: write a typed, tested Python function; build and run a two-endpoint API service; query the table a prompt will be built from.

    Core vocabulary for AI engineering

    No prerequisites - a good place to start.

    A small vocabulary covers most design discussions: a token is the unit models read and bill in; the context window is how many tokens one request can hold; inference is running a trained model; an embedding is a vector that represents meaning; retrieval-augmented generation (RAG) puts retrieved text into the prompt; tool calling lets a model request a function call; an agent is a loop where the model chooses its next action; an eval is a repeatable test of output quality; grounding means an answer is supported by supplied sources.

    Why it matters: Precise words prevent expensive misunderstandings - "the model knows our policy" (it does not; retrieval supplies it) or "the agent is accurate" (accurate on which evaluation set?).

    Example: "Cut the context to 6k tokens and re-run the retrieval eval" is a complete instruction to someone who knows the terms, and a puzzle to someone who does not.

    Watch out: Using "AI", "model", "agent" and "assistant" interchangeably in specifications. Check: explain each term in one sentence without using the other terms.

    You can now: define the ten core terms; spot a vague claim and rewrite it precisely; follow a design review without stopping for vocabulary.

    AI-assisted coding, used well

    AI

    Learn first: Python, APIs & SQL first

    The skill most learners skip, and the one teams now watch in week one: drafting with Copilot, Cursor or Claude and verifying what comes back. The workflow has four steps. Specify - write the function signature, the types and the test first, because a vague request gets confident wrong code. Draft - let the assistant write the implementation, the test cases, the migration, the docstring. Check - run the tests, read the diff line by line, look for the hallucinated API, the silent exception, the off-by-one. Own - you review it, you commit it, you explain it in the pull request. The assistant is never the author.

    Why it matters: 80% of new developers on GitHub use Copilot in their first week (GitHub, Octoverse 2025), and 66% of leaders say they would not hire someone without AI skills (Microsoft and LinkedIn, Work Trend Index). The same Stack Overflow survey that finds 84% of developers using or planning to use AI tools finds only 33% trusting their accuracy and 45% losing time debugging AI-written code (2025 Developer Survey). The hire is the person who gets the speed without the debt.

    Example: An assistant drafts a retrieval function that quietly drops documents over 8,000 tokens. The test with one long document catches it; the fix is yours; the test stays in the suite.

    Watch out: Committing generated code you cannot explain. Check: for every AI-drafted change, name the test that would have failed without it.

    You can now: run the specify-draft-check-own loop on a real change; spot the three common faults in generated code; explain an AI-drafted change in a pull request as your own.

    Which role to prepare for

    Learn first: The AI engineer job in 2026

    Aim at one title, in one kind of company, in one city; the preparation, the portfolio and the job search all get sharper. The titles overlap: an AI engineer composes existing models into product features, an ML engineer trains and serves models, a data scientist analyses and experiments. Postings mix the words, and the responsibilities tell you which one a role really is. The path depends on where you start: a fresher needs the foundations stage first, a developer switching in already has them. The market is the third sub-node: who hires, where, and what moves you up a band.

    Why it matters: AI engineer is LinkedIn's fastest-growing job in the US for 2025 and again for 2026, and the people moving into it most often come from software engineering, data science and full-stack roles (Jobs on the Rise 2026). The World Economic Forum lists AI and machine learning specialists among the three fastest-growing jobs to 2030 (Future of Jobs Report 2025). The role is growing fastest exactly where ordinary software engineers add the model layer.

    Example: AI engineer, SaaS product company, Bengaluru, retrieval and evaluation is a target. Something in AI produces a portfolio of notebooks nobody asked for and applications to roles that want a different person.

    Watch out: Preparing for the title in the posting instead of the responsibilities in it. Check: take three postings you want, highlight the verbs - build, evaluate, integrate, train - and count which dominate.

    You can now: name the three roles and their centres of gravity; choose the track that fits where you start; write your target as title, company type and city.

    AI engineer vs ML engineer vs data scientist

    No prerequisites - a good place to start.

    The titles overlap, but the centre of gravity differs. A machine learning engineer trains, tunes and serves models from data and owns training pipelines and model quality. A data scientist analyses data, designs experiments and builds predictive models to answer business questions. An AI engineer composes existing models - usually through APIs or open weights - into product features and owns prompts, retrieval, tools, evaluation, cost and safety around them.

    Why it matters: LinkedIn's fastest-growing job of 2025 and 2026 is the AI engineer, and the people moving into it most often come from software engineering, data science and full-stack roles, not from research (LinkedIn, Jobs on the Rise 2026). Job descriptions mix these words freely. Reading which responsibilities a posting actually lists tells you which skills will be tested and whether this roadmap fits the role.

    Example: A posting that asks for "PyTorch, distributed training and feature stores" is an ML engineering role; one that asks for "RAG, evals, tool calling and LLM observability" is an AI engineering role, even if both are titled "AI Engineer".

    Watch out: Assuming you must master model training before applying for applied AI roles. Check: highlight the verbs in three postings you want - build, evaluate, integrate, train - and count which dominate.

    You can now: explain the difference between the three roles; classify a job posting by its responsibilities; decide which optional training topics you actually need.

    Fresher path vs switcher path

    Optional on the Experienced track

    Learn first: The AI engineer job in 2026

    Freshers and career switchers start from different places. A fresher needs the foundations stage first - Python, Git, HTTP and testing - and should treat optional nodes as later reading. A developer switching in already has most foundations, so the fastest route is model behaviour, model APIs, evaluation, agents, security and production, using existing backend or frontend experience as an advantage.

    Why it matters: The median AI engineer hired in the US had 3.7 years of prior experience, most often in software engineering (LinkedIn, Jobs on the Rise 2026) - a switcher's foundations are the asset, a fresher's are the first thing to build. Following someone else's path wastes months. The track switch on this map changes which nodes are filled (focus now) and which are outlined (when you need them) for exactly this reason.

    Example: A fresher spends the first weeks writing small Python programs with tests and calling an HTTP API; a backend developer spends the same weeks building a retrieval feature with an evaluation set and a cost estimate.

    Watch out: Skipping foundations because model topics feel more exciting. Check: before the model stages, you should be able to write a typed Python function, test it with pytest and call a JSON API with error handling.

    You can now: choose the track that fits you; plan your first month on this roadmap; explain why outlined nodes can wait.

    Where the jobs are: sectors, cities and pay

    Learn first: AI engineer vs ML engineer vs data scientist

    The India market for applied AI engineers in 2026. The companies that hire are SaaS and product companies, the global capability centres of US and European firms, the AI practices of IT services companies, fintech, and AI-first startups. Bengaluru, Hyderabad, Pune, the Delhi region and Mumbai carry most of the openings, and remote and hybrid roles widen the map. Pay follows the kind of company more than the city: services firms start freshers lowest, product companies and capability centres pay about a band higher for the same skills, and a shipped feature with evaluation numbers is what moves a candidate across that line.

    Why it matters: Demand is real and recent. Generative-AI job postings in the US rose from 16,000 in 2023 to more than 66,000 in 2024, and prompt-engineering postings from 1,400 to nearly 6,300 (Stanford AI Index 2025, Lightcast data). In India, white-collar hiring ended 2025 up 13% in December with fresher hiring up 30% year on year in November (Naukri JobSpeak). Coursera's India salary guide, drawing on Glassdoor and Payscale, shows AI engineers earning clearly above general software roles at every level, with Delhi, Bengaluru and Mumbai ahead of Pune.

    Example: A fresher at an IT services firm who ships one retrieval feature with an evaluation set and a cost sheet on GitHub typically interviews at product companies a band higher within a year, without changing city.

    Watch out: Chasing the city instead of the company type and the proof. Check: for your target role, name two companies of each type that hire for it and what each asks for in its postings.

    You can now: name the five kinds of employer and what each pays for; explain why a shipped, measured feature moves pay more than a city move; pick three target companies.

    What companies actually expect

    Hiring

    Learn first: The AI engineer job in 2026

    Not a list of frameworks. In 2026 a team expects three things from an AI engineer it hires: that you can make a model-backed feature dependable, that you have proof of it, and that you work the way the team already works - with AI tools, under tests. The tests are a coding round, a practical build such as a small retrieval or extraction feature, and a design discussion about evaluation, cost, safety and failure. The day-to-day is specifications, versioned prompts, retrieval and tool integrations, evaluation cases from real failures, traces, cost and latency, and short decision notes. The proof is one shipped project with an evaluation set, a measured cost per request and a write-up - and, for a fresher, one recognised certification.

    Why it matters: 78% of organisations now use AI and 71% use generative AI, up from 55% and 33% two years earlier (Stanford AI Index 2025), and software-development postings mention AI 20% or more of the time (Indeed Hiring Lab, January 2026). Teams are not short of demos; they are short of people who can make them dependable.

    Example: Design a support assistant for 10,000 customers a day is answered with scope, retrieval and permissions, an evaluation plan, latency and cost per conversation, escalation to a human and rollout - not with a framework name.

    Watch out: Preparing a feature list of tools instead of evidence of judgment. Check: for a project you built, can you name one failure you found through evaluation, how you fixed it, and what it costs per request?

    You can now: describe the three things a team expects and the test that probes each; tell a demo from an engineered feature in one minute; list what your own proof is still missing.

    What hiring teams test now

    Learn first: Day-to-day responsibilities

    Interviews for applied AI roles usually combine a coding round, a practical exercise and a design discussion. The practical part is often a take-home or live task: build a small retrieval or extraction feature, then explain how you would measure it. Design rounds probe system thinking for LLM features - where retrieval fits, how to evaluate, how to bound an agent, what it costs, and what you do when the model is wrong or an input is malicious.

    Why it matters: 71% of leaders would rather hire a less experienced candidate with AI skills than a more experienced one without them (Microsoft and LinkedIn, Work Trend Index) - and more than 1.1 million public repositories already call an LLM SDK (GitHub, Octoverse 2025). Hiring teams can no longer separate candidates by who can call an API. They look for evidence of judgement: tests, evaluation numbers, cost estimates, failure handling and a clear explanation of trade-offs.

    Example: "Design a support assistant for 10,000 customers a day" is answered well by covering scope, retrieval and permissions, an evaluation plan, latency and cost per conversation, escalation to humans and rollout - not by naming a framework.

    Watch out: Preparing only by memorising framework APIs. Check: can you explain, for a project you built, one failure you found through evaluation and how you fixed it?

    You can now: describe the common interview formats; prepare a project walkthrough that shows judgement; answer an LLM system design prompt in a structured way.

    Day-to-day responsibilities

    Learn first: What an AI engineer builds

    A typical week mixes product engineering with measurement: turning a vague request into a specification, writing and versioning prompts, building retrieval or tool integrations, adding test cases from real failures, reading traces to find why an answer went wrong, tuning cost and latency, reviewing security questions, and explaining trade-offs to product managers and reviewers.

    Why it matters: 46% of developers distrust the accuracy of AI output and 45% say debugging AI-written code is time-consuming (Stack Overflow, 2025 Developer Survey) - which is why so much of the week is traces and evaluation cases. The parts that take the most time - debugging from traces, growing evaluation sets, writing decision notes - rarely appear in tutorials, yet they are what teams hire for and what interviews probe.

    Example: A bug report says the assistant quoted last year's refund policy. The engineer reproduces it, finds retrieval returned an archived document, adds a version filter, adds the case to the regression set and writes a two-line note on what changed and how it was verified.

    Watch out: Fixing a failure by editing the prompt without a test that proves the fix. Check: every bug fix should leave behind a new evaluation case that failed before and passes after.

    You can now: list the recurring tasks of the role; describe a debug-fix-verify loop for a model failure; write a short update that separates what you established from what you assume.

    Projects and proof that count

    Learn first: What hiring teams test now

    A recruiter scans for one recognised certification; the hiring team decides on one project. Have both, in that proportion. The certifications recruiters recognise are Microsoft's AI-102 Azure AI Engineer Associate, AWS's Certified AI Practitioner for the cloud-side vocabulary, and OneRoadmap's AI Engineer certification, which becomes verified evidence on your profile here. The project hiring teams believe is one feature on real documents or tickets - retrieval or extraction - with a README that runs, an evaluation set of fifty or more real questions with one failure found and fixed, abstention when the answer is not in the sources, and cost and latency per request in the write-up.

    Why it matters: LinkedIn's Skills on the Rise 2026 report puts it plainly: employers are looking less at job titles or degrees and more at what people can actually do. A certificate opens the screen; the project with numbers is what the hiring team can actually assess.

    Example: Answer questions about a 200-page employee handbook, say not in the handbook when it is not, and keep the cost per question under a set budget - with the evaluation results and the cost in the README. That one project answers every interview question about judgment.

    Watch out: A chatbot demo with no tests, a notebook that calls an API, a list of frameworks, or a claim of 100% accuracy - each one tells a reviewer the opposite of what you intend. Check: does your project page show an evaluation number, a failure you fixed, and a cost per request? If not, it is not finished.

    You can now: choose the certification that fits your target; specify a portfolio project a hiring team will believe; say which three items a reviewer's rubric credits first.

    Fresher: certification first, project second, then apply. Experienced: skip the beginner certificate; a published project in your domain is the signal.

    What gets you hired

    Learn first: The AI engineer job in 2026

    Foundations get you screened; a shipped feature with evaluation gets you interviewed; the explanation of one failure you found and fixed gets you hired. The three sub-nodes are what to build - the six product shapes almost every LLM feature falls into, and therefore the six portfolio ideas - the 90-day plan from foundations to a shipped, measured project and rehearsed interviews, and the proof stack you can build here: a scored certification, daily challenges and a readiness score employers can see.

    Why it matters: The World Economic Forum expects 39% of today's skills to change by 2030 (Future of Jobs Report 2025), and the model you learn on will be replaced within a year. Teams hire the engineer whose judgment - tests, evaluation, cost, failure handling - does not change when the model does, and that judgment is exactly what a shipped project and a scored assessment make visible.

    Example: Two candidates with the same coding score. One shows a chatbot demo; the other shows a retrieval feature with an evaluation set of fifty questions, one bug it caught, and a cost per query. The second gets the offer, and the interviewer can say why.

    Watch out: Collecting tutorials and tools instead of finishing one measured feature. Check: can a stranger run your project from its README and see its evaluation number inside ten minutes?

    You can now: say what gets a candidate screened, interviewed and hired; pick the product shape your first project will take; name the three pieces of proof you will build here.

    What an AI engineer builds

    No prerequisites - a good place to start.

    The products are familiar software with a model inside: question answering over company documents, classification and routing of tickets or emails, extraction of fields from invoices and contracts, drafting assistants, coding and data agents, voice front-ends and internal copilots. Each one combines ordinary code (APIs, databases, queues, auth) with model calls whose output must be constrained and verified.

    Why it matters: Organisations using generative AI went from 33% to 71% in two years (Stanford AI Index 2025), and almost all of it is these six shapes, not new research. Recognising the common product shapes lets you reuse patterns - retrieval for knowledge, structured output for extraction, tool calling for actions - instead of treating every request as a new research problem.

    Example: "Summarise every new support ticket and suggest a reply" is three components: a classifier with a fixed label set, a retrieval step over approved answers, and a drafting call whose output a human approves before it is sent.

    Watch out: Reaching for an agent when a single structured call would do. Check: for each feature, write down the input, the exact output format and who acts on it; if you cannot, the feature is not specified yet.

    You can now: name six common LLM product patterns; break a feature request into model calls and ordinary code; say which pattern a given request needs.

    The 90-day plan

    Learn first: Fresher path vs switcher path

    Twelve weeks from foundations to interview-ready, for somebody with two hours a day; a developer switching in compresses the first three weeks into one and spends the time on evaluation and production. Weeks one to three are foundations: typed Python with pytest, a small FastAPI service, SQL basics, Git with pull requests, finishing with a model call that returns a JSON schema and handles errors. Weeks four to seven are one shipped feature: retrieval or extraction over real documents, an evaluation set of fifty real questions, one failure found through it and fixed, abstention, and cost and latency per request in the README. Weeks eight and nine are agents and safety, bounded: one tool-calling loop with a step limit, a permission check and a readable trace, plus prompt-injection tests on your own feature. Weeks ten to twelve are the certification, the project write-up, the four interview moments rehearsed, and ten targeted applications rather than a hundred generic ones.

    Why it matters: The median AI engineer hired in the US had 3.7 years of prior experience, most often in software engineering (LinkedIn, Jobs on the Rise 2026). A fresher closes that gap only with proof a hiring team can check, and twelve weeks is enough to build one piece of it properly.

    Example: Week six. Monday build the evaluation set from real questions, Tuesday find the long-document bug, Wednesday fix it and keep the test, Thursday measure cost per query, Friday write the two-paragraph decision note.

    Watch out: Spending the twelve weeks on courses and arriving with nothing that runs. Check: at the end of week seven, can a stranger run your feature and read its evaluation number?

    You can now: lay out your own twelve weeks on a calendar; say what is finished at the end of each block; explain which block a switcher compresses and why.

    Your proof stack on OneRoadmap

    Learn first: The 90-day plan

    Everything this section asks you to prove can be proved here, and it becomes verified evidence on your profile that employers and the match score read. The AI Engineer certification is free, scored and the role credential: the questions a design round asks, graded by code against a rubric. Daily challenges are short, scored situations from the real job - a wrong answer in a trace, a take-home with four hours left - one capability each, with a streak that keeps judgment sharp. Role Readiness is your score out of 100 for the role, built from that evidence, with the gaps and what closes each. When a job fits, Ori prepares the application from the same evidence; you review and submit.

    Why it matters: LinkedIn's Skills on the Rise 2026 report finds employers looking less at titles or degrees and more at what people can actually do, and 71% of leaders prefer demonstrated AI skills to extra years of experience (Microsoft and LinkedIn, Work Trend Index). Evidence that was graded, not claimed, is what those employers can act on.

    Example: Certification done, a 20-day challenge streak and a readiness score of 70, plus one shipped feature on GitHub with its evaluation numbers - a profile a hiring team can trust before the first call.

    Watch out: Ticking nodes on this map and calling it progress; a tick verifies nothing. Check: which of the three pieces of proof here do you have, and which is the next one to earn?

    You can now: name the three kinds of evidence this platform scores; read your own readiness score and its gaps; say what Ori prepares from that evidence and what stays yours to do.

  2. 2. Engineering foundations first

    Python for AI engineering

    No prerequisites - a good place to start.

    Python is the default language for AI engineering: provider SDKs, evaluation tools, data libraries and most examples start there. You need everyday fluency rather than language trivia - functions, data structures, modules, virtual environments, type hints with validation, async calls for many concurrent requests, tests, error handling and logging.

    Why it matters: Most failures in LLM features are ordinary software failures: an unhandled timeout, a field that was None, a dependency that changed, a retry that never stops. Solid Python habits remove a whole class of bugs before any model is involved.

    Example: A script that classifies 5,000 tickets needs a validated input schema, concurrent requests with a limit, retries for rate-limit errors, a log line per failure and a test that runs on three fixed tickets without calling the real API.

    Watch out: Copying notebook code straight into a service. Check: your project installs from a pinned dependency file, passes its tests with one command and logs failures without printing secrets.

    You can now: set up a reproducible Python project; write typed, tested functions; call external APIs concurrently and safely.

    Fresher: take this stage slowly and finish small programs. Experienced: focus on async, Pydantic validation and testing with mocks.

    Python fundamentals

    Optional on the Experienced track

    No prerequisites - a good place to start.

    The fundamentals you will use daily: variables and core types, lists, dictionaries and sets, loops and comprehensions, functions with clear inputs and outputs, modules and imports, reading and writing files, and working with JSON. Classes are useful but secondary; most AI glue code is functions that transform dictionaries and strings.

    Why it matters: Model responses, API payloads and evaluation results are all nested dictionaries and lists. Being fluent with them is the difference between debugging in minutes and debugging in hours.

    Example: Load a JSON file of 50 customer questions, keep those under 500 characters, group them by category in a dictionary and write a summary file with counts per category.

    Watch out: Mutating a list or dictionary while looping over it, and silently ignoring missing keys. Check: run your function on an empty input, a normal input and an input with a missing field, and make each behaviour explicit.

    You can now: write small programs with functions and modules; manipulate nested JSON confidently; read a traceback and locate the failing line.

    Environments & dependencies

    Learn first: Python fundamentals

    Every project gets its own isolated environment and a declared list of dependencies with versions, so it installs the same way on your laptop, a teammate's machine and a server. The standard tools are the built-in venv module with pip and a requirements file, or a project manager such as uv or Poetry that writes a lock file of exact versions.

    Why it matters: Model SDKs and AI libraries release often and change behaviour between versions. An unpinned dependency can break a working feature overnight or make an evaluation impossible to reproduce.

    Example: Create a project with a pyproject file, add the provider SDK and pytest, commit the lock file, and confirm a fresh clone installs and runs the tests in one command.

    Watch out: Installing packages globally and relying on whatever versions happen to be there. Check: delete the environment, recreate it from the lock file and run the tests - if anything breaks, the dependencies were not really declared.

    You can now: create and activate an isolated environment; pin and upgrade dependencies deliberately; reproduce a project from a clean checkout.

    Type hints & Pydantic models

    Learn first: Python fundamentals

    Type hints document what a function expects and returns, and let your editor and a type checker catch mistakes before runtime. Pydantic turns those types into runtime validation: you declare a model class with fields, types and constraints, and it parses incoming data, coerces what it safely can and raises a clear error for the rest.

    Why it matters: Model output is untrusted data. Declaring the shape you need as a Pydantic model gives you one place to validate it, a JSON Schema to send to providers that support structured outputs, and a typed object for the rest of your code.

    Example: A TicketTriage model with category limited to five allowed values, priority as an integer from 1 to 3 and order_id optional; parsing a response with priority "urgent" fails with an error naming the field.

    Watch out: Validating only that the output is JSON. Check: write tests that feed invalid enums, out-of-range numbers, extra fields and missing fields, and confirm each is rejected or handled on purpose.

    You can now: annotate functions with types; define Pydantic models with constraints; generate a JSON Schema from a model and validate untrusted data against it.

    Async I/O & concurrency

    Learn first: Python fundamentals, HTTP, REST & JSON

    Model calls spend almost all their time waiting on the network. Python's asyncio lets one process keep many requests in flight: functions declared with async def are awaited, and helpers such as gather or task groups run them concurrently. A semaphore or a worker pool caps how many run at once so you respect rate limits.

    Why it matters: Processing 2,000 documents one call at a time can take hours; with bounded concurrency it takes minutes - without tripping the provider's rate limit or exhausting memory.

    Example: An evaluation run sends 300 test questions with at most 10 requests in flight, collects successes and failures separately, and retries only the failures after a short wait.

    Watch out: Forgetting to await a coroutine, which returns a coroutine object instead of a result, and unbounded gather calls that fire thousands of requests at once. Check: log the number of in-flight requests and confirm it never exceeds your limit.

    You can now: write async functions that call an HTTP API; run work concurrently with a limit; collect partial failures without losing successful results.

    Testing with pytest

    Learn first: Python fundamentals

    pytest runs plain functions whose names start with test, with assert statements as checks. Fixtures supply reusable setup such as a sample document or a fake client, and parametrisation runs one test over many cases. For code that calls a model, tests replace the network call with a stub that returns a recorded response, so they run fast, offline and for free.

    Why it matters: Tests keep the deterministic parts of an AI feature - parsing, validation, routing, permission checks, retries - correct while prompts and models change around them. They are also the habit that evaluation sets build on.

    Example: A parametrised test feeds five recorded model responses - valid, invalid enum, truncated JSON, empty, refusal - into your parser and asserts the exact handling for each.

    Watch out: Tests that call the live API, which are slow, cost money and fail randomly. Check: your test suite passes with networking disabled.

    You can now: write and run pytest tests; use fixtures and parametrisation; stub external API calls with recorded responses.

    Errors, logging & debugging

    Learn first: Python fundamentals

    Exceptions should be handled where you can do something useful - retry, fall back, return a clear error - and allowed to surface elsewhere. Structured logging records what happened with fields you can search: request id, model, latency, token counts, outcome. A debugger or a few well-placed log lines beat guessing.

    Why it matters: Model features fail in new ways: timeouts, rate limits, refusals, truncated output, invalid JSON. Without specific handling and useful logs, every one of them looks like "the AI is broken".

    Example: Catch the provider's rate-limit error and retry with backoff; catch a validation error and record the raw output id for review; let an unexpected error propagate with the request id attached.

    Watch out: A bare except that swallows every error, and logs that contain API keys or full personal data. Check: trigger each failure type on purpose and confirm the log tells you what happened without exposing secrets.

    You can now: handle specific exceptions deliberately; write structured log lines with request ids; debug a failing call from its logs alone.

    Software engineering basics

    Learn first: Python fundamentals

    AI features live inside ordinary services, so the usual engineering basics apply: version control, HTTP and JSON, a small API service, a database you can query, configuration and secrets kept out of code, and AI coding assistants used with review. This is the layer that makes a prototype into something a team can run.

    Why it matters: Reviewers and interviewers judge AI projects on engineering hygiene as much as on model results. A clean repository with tests and a working API endpoint is more convincing than a notebook.

    Example: A retrieval prototype becomes a service: a FastAPI endpoint, request validation, an environment variable for the API key, a PostgreSQL table for documents, a pull request with tests and a short README.

    Watch out: Building the whole feature in one notebook with keys pasted inline. Check: someone else can clone the repository, set two environment variables and run it.

    You can now: work in a Git repository with pull requests; build and call a JSON API; keep configuration and secrets out of source code.

    Fresher: do every node here. Experienced: check the secrets and AI-assistant nodes, then move on.

    Git & pull requests

    Optional on the Experienced track

    No prerequisites - a good place to start.

    Git records every change to a project; branches let you work on a change in isolation; a pull request lets someone review the change before it merges. Small, focused commits with clear messages make history readable, and a .gitignore keeps environments, data files and secrets out of the repository.

    Why it matters: Prompts, evaluation sets and configuration are code in an AI project. Versioning them in Git is how you know which prompt produced which result and how you roll back a bad change.

    Example: A branch changes the system prompt and adds four evaluation cases; the pull request description shows the before-and-after evaluation scores and links the bug it fixes.

    Watch out: Committing a .env file with an API key - once pushed, treat the key as leaked and rotate it. Check: run git status before every commit and keep secrets in a git-ignored file.

    You can now: branch, commit and open a pull request; write a useful commit message; review a diff and recover from a bad change.

    HTTP, REST & JSON

    Learn first: Python fundamentals

    Model APIs are HTTP APIs: a client sends a request with a method, URL, headers and a JSON body, and gets back a status code, headers and a JSON body. Status codes carry meaning - 2xx success, 400 a bad request, 401 or 403 an authentication problem, 429 a rate limit, 5xx a server-side failure - and each calls for different handling.

    Why it matters: Reading raw requests and responses is how you debug an SDK that misbehaves, call a provider without an SDK, and build your own endpoints for others to use.

    Example: A call returns 429 with a retry-after header; the right response is to wait and retry, not to change the prompt. A 401 means the key is wrong or missing, not that the model is down.

    Watch out: Treating every non-200 response the same way. Check: list how your client handles 400, 401, 404, 429, 500 and a network timeout, and test at least three of them.

    You can now: read and write JSON payloads; interpret HTTP status codes and headers; call a REST API with curl or Python and handle errors by type.

    Build a small API service

    Learn first: HTTP, REST & JSON, Type hints & Pydantic models

    Most AI features are exposed as an endpoint: a request comes in, it is validated, the model and any retrieval or tools are called, the result is validated and returned. A framework such as FastAPI gives you routing, request validation from type hints, automatic API docs and async support in a few lines.

    Why it matters: A service boundary is where you enforce input limits, authentication, timeouts and output validation - the controls that keep a model feature safe and affordable.

    Example: POST /triage accepts a ticket of at most 4,000 characters, calls the model with a 20-second timeout, validates the JSON against a schema and returns either the triage or a clear error with a request id.

    Watch out: Returning raw model text to clients and letting requests run with no timeout. Check: send an oversized input, an empty input and a request that times out, and confirm each gets a defined response.

    You can now: build a small JSON API with validation; add timeouts and error responses; document and test the endpoint.

    SQL & the data you query

    Learn first: Python fundamentals

    Real features read real data: orders, tickets, documents and user permissions usually live in a relational database. You need SELECT with filters, joins, aggregates and parameterised queries, plus enough PostgreSQL to store documents and metadata - and, with the pgvector extension, embeddings next to them.

    Why it matters: Retrieval quality and access control often depend on ordinary SQL - filtering documents by tenant, freshness or permission before any vector search happens.

    Example: Fetch the ten most recent approved policy documents for one tenant with a parameterised query, then pass only those to the retrieval step.

    Watch out: Building SQL by pasting user or model text into a query string, which invites injection. Check: every query that includes external input uses parameters, never string formatting.

    You can now: write filtered, joined and aggregated queries; use parameterised queries from Python; model a simple documents table with metadata.

    Config, env vars & secrets

    Learn first: Environments & dependencies, Git & pull requests

    Configuration that changes between environments - API keys, model names, endpoints, limits - belongs outside the code, usually in environment variables or a secrets manager. Keys are scoped to what they need, rotated when exposed, and never printed, logged or sent to a model.

    Why it matters: Leaked model API keys are spent by strangers within hours. Keeping configuration out of code also lets you switch models or limits without a code change.

    Example: The service reads the provider key and the model name from environment variables at start-up, fails fast with a clear message if either is missing, and logs only the model name.

    Watch out: Hard-coding keys in notebooks, committing .env files and pasting keys into prompts or bug reports. Check: search your repository history for key-like strings and set a spending limit in the provider console.

    You can now: load configuration from environment variables; keep secrets out of code, logs and prompts; respond correctly to a leaked key.

    AI coding assistants, used well

    AI

    Learn first: Testing with pytest, Git & pull requests

    Coding assistants and agentic tools - editor copilots, chat assistants and terminal agents - draft code, tests and refactors quickly. Used well, they speed up the boring parts; used carelessly, they add plausible code nobody has read. The skill is directing them precisely and reviewing everything they produce.

    Why it matters: Teams now expect engineers to work with assistants, and they judge the result like any other contribution: does it run, is it tested, is it secure, does the author understand it?

    Example: Ask the assistant for a retry wrapper with exponential backoff and a test; read the code, notice it retries on 400 errors as well as 429, fix the condition and add a test case for it.

    Watch out: Accepting code you cannot explain, and pasting secrets or private data into a tool your company has not approved. Check: for every generated change, run the tests and explain each line in review.

    You can now: give an assistant a precise, testable task; review and correct generated code; decide what data is safe to share with a coding tool.

  3. 3. Understand the models you build on

    How LLMs work, practically

    Learn first: Core vocabulary for AI engineering

    You do not need the maths of training to build with language models, but you do need an accurate mental model of their behaviour: they read text as tokens, predict likely next tokens, are limited by a context window, vary their output with sampling settings, can be asked to reason at extra cost, and produce fluent text whether or not it is true.

    Why it matters: Almost every design decision - how long a prompt can be, why an answer changed between runs, why retrieval is needed, why outputs must be validated - follows from these few properties.

    Example: An assistant gives a different refund deadline on each run. Knowing about sampling and missing context, you look for the policy text in the prompt (absent), add retrieval of the current policy and lower the temperature, then confirm with a repeated-run test.

    Watch out: Explaining model behaviour with human intuitions ("it understood", "it forgot"). Check: for any surprising output, ask what tokens the model actually saw and what settings produced the sample.

    You can now: explain generation, tokens, context and sampling in plain terms; predict the cost and limits of a request; reason about why an output went wrong.

    Fresher: read every subtopic. Experienced: focus on tokens, reasoning models and failure modes.

    Next-token prediction

    Learn first: Core vocabulary for AI engineering

    A large language model is a neural network trained to predict the next token of text from the tokens before it. Generation repeats that step: the model produces a probability for every possible next token, one is chosen, it is appended to the input, and the process continues until a stop condition. Pretraining on huge text corpora gives broad language ability; post-training on instructions and feedback turns that into a helpful assistant.

    Why it matters: Because the model produces plausible continuations rather than looking facts up, fluent answers can be wrong, the same prompt can give different outputs, and knowledge stops at the training data unless you supply more in the prompt.

    Example: Asked for a company's current refund window, a model with no supplied policy produces a typical-sounding answer such as "30 days", because that is a likely continuation, not because it checked anything.

    Watch out: Treating a confident answer as a retrieved fact. Check: for any factual answer your product gives, identify the source text in the prompt that supports it; if there is none, the answer is a guess.

    You can now: describe generation step by step; explain pretraining versus post-training; explain why fluency is not evidence of correctness.

    Tokens & tokenisation

    Learn first: Next-token prediction

    Models do not read characters or words; a tokenizer splits text into tokens - common words, word pieces, punctuation, spaces and bytes. In English a token averages roughly three to four characters, but code, numbers and many non-English scripts use more tokens for the same meaning. Providers count, limit and bill everything in tokens: input, output and, for reasoning models, hidden reasoning tokens.

    Why it matters: Tokens decide cost, latency and whether a request fits in the context window. Estimating them is the first step in every budget and every long-document design.

    Example: A 3,000-word policy document is roughly 4,000 tokens in English; the same content in Hindi can need noticeably more. Count it with the provider's tokenizer or token-counting endpoint before choosing how much to include.

    Watch out: Estimating cost from word counts or from one provider's tokenizer for another provider's model. Check: compare your estimate with the usage numbers the API returns for a few real requests.

    You can now: explain what a token is; count tokens for a given model; estimate the token cost of a prompt and its expected output.

    Context windows & knowledge cut-offs

    Learn first: Tokens & tokenisation

    The context window is the maximum number of tokens a model can process in one request - instructions, conversation history, retrieved documents and tool results together - and the output has its own separate maximum. The knowledge cut-off is the date after which the model saw no training data. Model pages also list capabilities such as image input, tool calling, structured output and reasoning settings, which differ between models.

    Why it matters: These numbers rule designs in or out before any benchmark does: a 300-page contract may not fit, recent events need retrieval, and a model without image input cannot read a scanned invoice.

    Example: A model with a 200,000-token window and an 8,000-token output limit can read a long report but cannot rewrite all of it in one response; you design a section-by-section flow instead.

    Watch out: Assuming the model knows anything after its cut-off, and assuming that a long window means the model uses every part of it equally well. Check: read the model page for window, output limit, cut-off and supported inputs before you design.

    You can now: find and interpret a model's limits; decide whether content fits; tell when you need retrieval rather than a bigger window.

    Sampling: temperature & top-p

    Learn first: Next-token prediction

    After the model scores possible next tokens, a sampler picks one. Temperature reshapes the probabilities - lower values make the top choices dominate, higher values spread choice out. Top-p keeps only the smallest set of tokens whose probabilities add up to p. Maximum output tokens and stop sequences end generation. Several current hosted models, reasoning models especially, fix these settings or reject requests that change them and offer an effort setting instead; sampling settings remain everyday tools for many other models, including most open-weight ones, so check each model's documentation.

    Why it matters: Extraction and classification want consistent outputs; brainstorming and drafting can tolerate variety. Choosing settings deliberately reduces random failures.

    Example: A ticket classifier runs at low temperature with a tight output limit and a fixed label set; a slogan generator runs higher and asks for several options.

    Watch out: Believing temperature 0 makes outputs identical every time - providers do not guarantee that, and batching and hardware can still change results. Check: run the same input ten times and measure how often the answer changes before relying on it.

    You can now: explain temperature and top-p; choose settings per task; measure output variability instead of assuming it.

    Transformer & attention intuition

    Learn first: Next-token prediction

    Modern language models are transformers. Each token becomes a vector (an embedding); layer by layer, an attention mechanism lets every position weigh information from other positions, so meaning is built from context; a final step turns the last position's vector into next-token probabilities. During generation, a key-value cache stores earlier computation so each new token is cheaper.

    Why it matters: This intuition explains practical facts: longer inputs cost more compute and memory, models are sensitive to wording and order, and serving long contexts for many users needs a lot of GPU memory for the cache.

    Example: Doubling the conversation history roughly doubles the input tokens billed and grows the cache the server must hold, which is why long chats get slower and costlier per turn.

    Watch out: Trying to learn the full mathematics before building anything. Check: you can explain attention, embeddings and the KV cache in two sentences each, and connect each to cost or latency.

    You can now: describe a transformer at a high level; explain what attention does; relate context length to compute, memory and cost.

    Reasoning models & effort settings

    2026

    Learn first: Next-token prediction, Tokens & tokenisation

    Reasoning models are trained to work through a problem in intermediate steps before giving a final answer. Providers expose a setting that trades thinking against speed and cost - an effort level or a budget for thinking tokens - and the thinking tokens are typically billed as output tokens even when they are not returned in full.

    Why it matters: Extra reasoning helps on multi-step problems such as planning, code changes and tricky analysis, but it adds latency and cost and often adds nothing for simple extraction or classification.

    Example: A routing classifier gets the same accuracy at low effort and high effort on your evaluation set, so you use low; a code-repair task improves from 62% to 81% at higher effort, so you pay for it there and nowhere else.

    Watch out: Turning reasoning up everywhere "to be safe". Check: compare accuracy, latency and cost per task at two effort settings on the same evaluation set and keep the cheapest one that passes.

    You can now: explain what reasoning models do differently; configure effort or thinking budgets; decide per task whether reasoning is worth its cost.

    Hallucination & other failure modes

    Learn first: Next-token prediction

    Models produce confident falsehoods (hallucinations), use outdated knowledge, change answers when a prompt is reworded, agree too readily with the user, struggle with exact arithmetic and counting without tools, refuse harmless requests or comply with harmful ones, and follow instructions hidden inside documents they were only meant to read.

    Why it matters: Each failure mode has a known engineering mitigation - retrieval and citations, tools for calculation, validation, evaluation sets, guardrails, human review - but only if you expect it.

    Example: An assistant states a delivery date that appears nowhere in the order data. The fix is not a stronger instruction; it is supplying the order record, requiring the date to come from it, and returning "unknown" when it is missing.

    Watch out: Patching each failure with another sentence in the prompt. Check: for every failure you see, name its type and the structural control that prevents it, then add a test case.

    You can now: name the common failure modes; match each to a mitigation; design features that stay useful when the model is wrong.

    Choosing models & inference

    Learn first: How LLMs work, practically

    Model choice is an engineering decision, not a loyalty. You choose between providers, model sizes and hosting options by measuring them on your own task: first the quality bar, then latency and cost, then data handling and operational constraints. The answer is often different models for different steps.

    Why it matters: The largest model is rarely the right default; the smallest model that passes your evaluation usually saves most of the cost, and pinning versions keeps that decision valid.

    Example: A pipeline uses a small fast model to classify tickets, a larger model only for the 15% that need a drafted reply, and pins both model versions so a provider update cannot silently change results.

    Watch out: Picking a model from a leaderboard or a launch post. Check: every model choice in your project is backed by a run of your own evaluation set with quality, latency and cost recorded.

    You can now: shortlist models for a task; compare them on your own data; justify a choice in a short decision note.

    The model landscape

    Learn first: Context windows & knowledge cut-offs

    The landscape changes every few months, but its shape is stable: frontier models from major labs offered through their own APIs; the same and other models offered through cloud platforms; smaller and cheaper models for high-volume tasks; open-weight model families you can download, run and fine-tune; and specialised models for embeddings, speech, images and reranking. Each provider publishes a models page with current names, limits, prices and deprecations.

    Why it matters: Knowing the categories lets you ask the right questions about a new release: which category is it, what does it cost, where can it run, and what is it licensed for?

    Example: A team in a regulated industry uses a frontier model through its existing cloud platform for data-residency reasons, plus an open-weight embedding model it hosts itself.

    Watch out: Memorising model names - they date quickly. Check: you can find any provider's current model list, prices and deprecation page within a minute.

    You can now: describe the categories of models and where they are offered; read a provider models page; place a new release in the landscape.

    Quality, latency & cost trade-offs

    Learn first: The model landscape, Success criteria & what to measure

    For each step in your feature, define the minimum quality it must reach on your evaluation set, the latency users will tolerate and the cost per request you can afford. Then test candidate models and settings against all three. Larger models usually improve quality and cost more; smaller models are faster and cheaper; reasoning settings, prompt length and output length move all three.

    Why it matters: A model that is 3% more accurate but four times the price, or two seconds slower in a chat interface, can be the wrong choice. Writing the trade-off down makes the decision reviewable.

    Example: Three models score 91%, 93% and 94% on the evaluation set; the bar is 92%. The 93% model is the cheapest that passes, so it ships, and the 94% model becomes the fallback for low-confidence cases.

    Watch out: Comparing models on different prompts, different datasets or a handful of examples. Check: run every candidate on the same frozen set with the same settings and report quality, p95 latency and cost per 1,000 requests together.

    You can now: set quality, latency and cost targets; run a fair model comparison; choose and justify the cheapest model that meets the bar.

    Reading benchmarks with care

    Learn first: The model landscape

    Public benchmarks test narrow skills - exam questions, maths, code, long-context recall - and leaderboards rank models on them or on human preference votes. They are useful signals of general capability, but test questions can leak into training data, scores can be reported under favourable settings, and a benchmark rarely matches your users' requests.

    Why it matters: A model that leads a reasoning benchmark may still be worse at your extraction task, your language mix or your formatting rules.

    Example: Two models are a point apart on a popular leaderboard; on your 200-ticket evaluation set, which includes Hinglish messages, one is eleven points better.

    Watch out: Quoting a benchmark score as evidence that your product is accurate. Check: before trusting a benchmark, read what it measures and how, and confirm the result on your own evaluation set.

    You can now: explain what common benchmarks measure; spot contamination and cherry-picking risks; use benchmarks to shortlist and your own evals to decide.

    Hosted APIs vs self-hosting

    Learn first: The model landscape

    You can call a model through the provider's API, through a cloud platform that hosts many models, through an inference provider that serves open-weight models, or run open-weight models yourself on your own GPUs. Hosted APIs give the strongest models with no infrastructure; self-hosting gives control over data, versions and unit cost at high volume, at the price of operating GPUs, scaling and upgrades.

    Why it matters: Data-handling rules, latency requirements, request volume and budget decide the right option - and the decision is expensive to reverse.

    Example: A startup prototypes on a hosted API; once it serves millions of simple classification calls a day, it moves that one step to a self-hosted small open-weight model and keeps the hosted API for complex replies.

    Watch out: Self-hosting to save money before measuring volume, utilisation and engineering time. Check: estimate monthly cost both ways at your real volume, including GPU idle time and the people needed to run it.

    You can now: list the hosting options; compare them on data control, quality, latency, cost and effort; recommend an option for a given workload.

    Model versions & deprecations

    Learn first: The model landscape

    Providers release dated model snapshots and moving aliases that point at the latest one, and they retire old models on published schedules. Pinning an exact version in production keeps behaviour stable; an alias can change under you. Every upgrade is a change to your system and deserves the same evaluation as a code change.

    Why it matters: A silent model change can shift formatting, refusal behaviour or accuracy overnight, and a retired model breaks a feature on a known date if nobody planned the migration.

    Example: The team pins a dated snapshot, subscribes to the provider's deprecation notices, and when a new version arrives runs the full evaluation set, compares results and only then changes the configured model name.

    Watch out: Using an alias such as "latest" in production and discovering the change from user complaints. Check: your configuration names an exact model version, and you know the retirement date of every model you use.

    You can now: distinguish snapshots from aliases; pin models in configuration; plan and evaluate a model upgrade.

  4. 4. Call models like a backend engineer

    Model APIs

    Learn first: HTTP, REST & JSON, Errors, logging & debugging

    Model APIs share a common anatomy across providers: you send instructions and messages, optional tools and an output format, plus settings such as the model name and output limit; you get back content, a stop reason and usage numbers. On top of that sit streaming, structured outputs, tool calling, error codes, rate limits and billing - the surface you build every feature on.

    Why it matters: Mastering one provider's API properly makes every other provider quick to learn, and the reliability details - timeouts, retries, validation, budgets - matter more than which SDK you use.

    Example: A thin client module wraps the provider SDK: it sets the model from configuration, applies a timeout, retries rate-limit errors with backoff, validates the output against a schema and logs usage per request.

    Watch out: Scattering raw SDK calls throughout a codebase. Check: all model calls go through one module that you can test with a stub and reconfigure in one place.

    You can now: make a model call with system instructions, messages and settings; read the response, stop reason and usage; wrap calls in a tested client.

    Fresher: build the client yourself once. Experienced: concentrate on structured outputs, tool calling and error handling.

    Request anatomy: messages & roles

    Learn first: HTTP, REST & JSON, Tokens & tokenisation

    A request carries instructions (a system or developer message that sets behaviour), the conversation as alternating user and assistant messages, and settings. The response contains the model's content, a stop reason - finished, hit the output limit, wants to call a tool, refused - and token usage. Most APIs are stateless, so you resend the relevant history each turn; some also offer server-side conversation state.

    Why it matters: Understanding roles and stop reasons explains common bugs: instructions placed in the wrong message, truncated answers treated as complete, and runaway costs from resending an ever-growing history.

    Example: A response stops with the "maximum tokens" reason halfway through a JSON object. The correct handling is to treat it as incomplete - raise the limit or shorten the task - not to parse what arrived.

    Watch out: Ignoring the stop reason. Check: your client inspects the stop reason on every response and has explicit handling for length limits, tool calls and refusals.

    You can now: build a multi-turn request with roles; read a response's content, stop reason and usage; manage conversation history deliberately.

    SDKs, raw HTTP & compatible APIs

    Learn first: Request anatomy: messages & roles

    Official SDKs wrap the HTTP API with types, retries and streaming helpers, and are the default choice. Raw HTTP is useful for debugging and for languages without an SDK. Many providers and local servers also implement an OpenAI-compatible API, so one client can talk to several back ends by changing the base URL and model name - although features beyond the basics often differ.

    Why it matters: Knowing what the SDK does for you - and what it hides - helps when a request fails, when you switch providers, or when you need a feature the abstraction does not expose.

    Example: The same client code calls a hosted model in production and a local open-weight model through an OpenAI-compatible endpoint in tests, switched by two environment variables.

    Watch out: Assuming compatible means identical: structured output, tool calling and streaming details vary. Check: run your own small contract test suite against every back end you claim to support.

    You can now: use an official SDK and make the same call with raw HTTP; point a client at a compatible endpoint; test which features a back end really supports.

    Structured outputs & JSON Schema

    Learn first: Request anatomy: messages & roles, Type hints & Pydantic models

    When another program consumes the output, ask for structure, not prose. Providers offer structured-output modes that constrain generation to a JSON Schema you supply, and strict tool definitions that do the same for tool arguments. A schema guarantees shape, not truth: you still validate values and business rules in code, and handle refusals and incomplete responses.

    Why it matters: Parsing free text is the most common source of brittle AI integrations. A schema plus validation turns a model call into a function with a typed return value.

    Example: An invoice extractor defines vendor, invoice date, currency limited to an enum and line items as an array of objects; the response is parsed into a Pydantic model and the line totals are checked against the invoice total in code.

    Watch out: Valid JSON that is wrong - an invented order id, a negative amount, a date in the future. Check: after schema validation, assert the business rules that matter and route failures to a defined path.

    You can now: write a JSON Schema for a task; use a provider's structured-output mode; validate both shape and business rules in code.

    Streaming responses

    Learn first: Request anatomy: messages & roles

    Streaming returns output in small chunks as it is generated, usually over server-sent events, instead of one response at the end. Users see the first words quickly, which matters for chat; your code must assemble chunks, handle tool-call and usage events that arrive mid-stream, and cope with a stream that stops early.

    Why it matters: Streaming improves perceived latency - time to first token - without making the full response faster. It also complicates validation, because you cannot validate structured output until it is complete.

    Example: A chat interface streams text to the user, while a background extraction call does not stream at all, because nothing can act on half a JSON object.

    Watch out: Claiming a feature got faster because it streams. Check: measure time to first token and total completion time separately, and validate structured output only after the final chunk.

    You can now: consume a streamed response; render partial text safely; measure and report both latency numbers.

    Tool (function) calling

    Learn first: Structured outputs & JSON Schema

    Tool calling lets a model ask your code to do something. You describe each tool with a name, a description and a JSON Schema for its arguments; the model may respond with a tool call instead of text; your code validates the arguments, checks permissions, runs the function and sends the result back; the model then continues. Models can request several tools at once.

    Why it matters: Tools connect models to live data and actions - order lookups, searches, calculations, ticket updates. They are the building block of every agent, and also where most real-world risk lives.

    Example: A lookup_order tool takes an order id; your code checks that the order belongs to the signed-in customer before returning it, and returns a clear error message the model can relay if it does not.

    Watch out: Executing whatever arguments the model produces. Check: every tool validates its arguments and enforces authorisation in code, exactly as an API endpoint would for an untrusted client.

    You can now: define tools with schemas; run a tool-calling round trip; enforce validation and permissions in the tool, not the prompt.

    Timeouts, retries & backoff

    Learn first: HTTP, REST & JSON, Errors, logging & debugging

    Model calls fail in predictable ways: network errors, timeouts, rate limits (HTTP 429), overloaded or failing servers (5xx) and invalid requests (400). Retry only the transient ones, wait longer after each attempt (exponential backoff) with random jitter, honour any retry-after header, cap the number of attempts and set an overall deadline. Never retry an action with side effects unless it is idempotent.

    Why it matters: Without bounded retries a brief provider incident becomes an outage of yours, a retry storm, or a duplicated refund.

    Example: A client retries 429 and 5xx up to four times with backoff and jitter within a 30-second deadline, fails fast on 400 and 401, and logs each attempt with the request id.

    Watch out: Infinite retry loops and retrying requests that can never succeed. Check: simulate a 429, a 500, a timeout and a 400 in tests and assert the number of attempts and the final outcome for each.

    You can now: classify errors as retryable or not; implement backoff with jitter and limits; protect side-effecting calls from duplicate execution.

    Usage, rate limits & budgets

    Learn first: Tokens & tokenisation, Request anatomy: messages & roles

    Every response reports usage - input tokens, output tokens and, where relevant, cached and reasoning tokens - and providers price input and output separately. Accounts have rate limits on requests and tokens per minute that rise with usage tiers. Provider consoles let you set spending limits and alerts, and your own code can enforce budgets per user, feature or job.

    Why it matters: Cost is a design constraint: a feature that costs more per user than it earns cannot ship, and a runaway loop can spend a month's budget in an hour.

    Example: Each request logs its token usage and computed cost; a daily job sums cost per feature, and the agent feature stops after a per-task budget of a fixed number of tokens.

    Watch out: Estimating cost once and never measuring it. Check: compare your estimate with logged usage after the first hundred real requests, and confirm a spending limit is set in the console.

    You can now: read usage from responses; compute cost per request and per feature; respect rate limits and enforce budgets.

    Playgrounds & consoles

    Optional on the Experienced track

    Learn first: Request anatomy: messages & roles

    Provider consoles include a playground where you can try prompts, switch models and settings, attach tools or schemas, and export the request as code. They are the fastest way to explore an idea, compare a few variations side by side and learn what a parameter does.

    Why it matters: A few minutes in a playground can rule out an approach before you write code - but results from three hand-picked inputs are anecdotes, not evidence.

    Example: You test a classification prompt on five tickets in the playground, export the request as Python, then move the prompt into your repository with twenty test cases before changing anything else.

    Watch out: Tuning a prompt in the playground until it works on the examples in front of you, then shipping it. Check: anything you keep from a playground goes into version control with tests within the same day.

    You can now: use a provider playground to explore models and settings; export a working request to code; move experiments into a tested workflow.

  5. 5. Prompts are code: version and test them

    Prompt engineering

    Learn first: Request anatomy: messages & roles

    A prompt is the specification a model works from: what role it plays, what task it does, what information it may use, what rules it must follow and exactly what to return. Good prompts are specific, structured and tested against representative inputs - and they are treated like code: templated, versioned and evaluated before release.

    Why it matters: Prompt quality still moves results more cheaply than any other lever, but a prompt tuned by feel on five examples breaks on real users. The engineering is in the testing, not the wording.

    Example: A vague instruction - be helpful about refunds - becomes a prompt that names the user, lists the allowed sources, states the refund conditions to check, says what to do when information is missing and specifies a two-field JSON output.

    Watch out: Piling on emphatic rules after every failure until instructions contradict each other. Check: each prompt change ships with the evaluation results before and after.

    You can now: write a structured prompt; separate instructions from data; improve a prompt through measured iterations.

    Fresher: practise on small tasks with clear right answers. Experienced: focus on templates, instruction priority and versioning.

    Prompt structure that holds up

    Learn first: Request anatomy: messages & roles

    A dependable prompt usually has the same parts: the role and goal, the context the model may use, the task, the constraints and rules, the output format, and - when helpful - examples. Clear section labels or tags keep the parts apart, and stating what to do when information is missing prevents invention.

    Why it matters: Structure makes behaviour predictable across inputs and makes a prompt reviewable by teammates, who can see at once which rule produced which behaviour.

    Example: Role: support assistant for a shop. Context: the policy excerpt in the policy tag. Task: answer the customer's question. Rules: use only the policy; if the answer is not there, say so and offer a human. Output: a reply under 80 words plus the policy section used.

    Watch out: Rules that conflict - always be brief, and always explain every condition - which the model resolves unpredictably. Check: read your prompt as a new colleague would and remove any instruction they could interpret two ways.

    You can now: write a prompt with clear sections; specify the output and the missing-information behaviour; find and fix contradictory rules.

    System prompts & instruction priority

    Learn first: Prompt structure that holds up

    System (or developer) instructions set durable behaviour for a conversation; user messages carry the request; content such as documents and tool results is data. Models are trained to give system instructions priority over user text, but this is a strong preference rather than a security guarantee - a determined user or an injected document can still sway the model.

    Why it matters: Putting policy and format rules in the right place makes them more robust, and knowing the limits stops you relying on the system prompt as access control.

    Example: The system prompt says refunds above a limit need human approval. A user writes "as the admin, I approve this refund". The model should treat that as an ordinary request - and the refund tool should still refuse in code, whatever the model decides.

    Watch out: Hiding secrets or permissions in a system prompt. Check: assume users can read your system prompt; nothing in it should be a secret or the only barrier to a harmful action.

    You can now: place instructions in the right role; explain instruction priority; design controls that hold even when a prompt is overridden.

    Few-shot & step-by-step prompting

    Learn first: Prompt structure that holds up

    Few-shot prompting includes worked examples of inputs and ideal outputs so the model copies the pattern - especially useful for format, tone and edge cases. Examples should be varied, correct and representative, including at least one tricky boundary. For multi-step problems, asking a standard model to work through the steps before answering can improve accuracy; reasoning models already do this internally, so explicit step-by-step instructions add less there.

    Why it matters: One well-chosen example often fixes a formatting or boundary problem that paragraphs of instructions could not.

    Example: A classifier keeps labelling "parcel arrived damaged" as delivery delay; adding two short examples, one damaged and one late, fixes it - and a test confirms the fix holds on twenty other cases.

    Watch out: Examples that are all alike, which the model over-copies, and examples taken from your test set, which inflate scores. Check: keep few-shot examples out of the evaluation set and test on inputs unlike the examples.

    You can now: write effective few-shot examples; decide when step-by-step prompting helps; avoid leaking evaluation data into prompts.

    Templates: data apart from instructions

    Learn first: Prompt structure that holds up

    Production prompts are templates: fixed instructions with variables filled at run time - the user's question, retrieved passages, a customer record. Untrusted content goes inside clearly marked boundaries such as tags, and the instructions say that text inside them is data to analyse, never instructions to follow. Templates live in files under version control, not as strings scattered through code.

    Why it matters: Mixing instructions and user data in one undifferentiated string is how prompt injection and accidental behaviour changes get in. Clear boundaries make both less likely and make prompts testable.

    Example: The template wraps the customer's message in a customer_message tag and the policy in a policy tag; a test inserts "ignore previous instructions" into the customer message and asserts the output still follows policy.

    Watch out: Building prompts with string concatenation and inserting raw documents without delimiters. Check: every variable in a template is delimited, length-limited and covered by a test with hostile content.

    You can now: build prompt templates with variables; delimit untrusted data; test templates with adversarial inputs.

    Prompt versioning & iteration

    Learn first: Prompt structure that holds up, Git & pull requests

    Treat each prompt as a versioned artefact: stored in the repository, identified by a version, changed one variable at a time, and evaluated on the same fixed set before and after every change. Record what changed, why and what the results were, so a regression can be traced to a specific edit and rolled back.

    Why it matters: Prompt changes have side effects - a fix for one complaint quietly breaks another case. Only a stable evaluation set and recorded results show the trade-off.

    Example: Version 7 shortens replies; the evaluation shows tone improved but two refund-condition cases now fail. You keep version 6, add a length rule that preserves conditions as version 8, and it passes both sets.

    Watch out: Comparing prompt versions on different examples, or changing the prompt, model and temperature at once. Check: each pull request that edits a prompt includes before-and-after scores on the same frozen set.

    You can now: version prompts alongside code; run controlled prompt experiments; roll back a prompt regression with evidence.

    Context engineering

    Learn first: Prompt structure that holds up, Context windows & knowledge cut-offs

    Context engineering is deciding, at every model call, exactly which tokens the model sees: instructions, retrieved documents, tool results, memory, conversation history and examples. The context window is a budget, and every token in it should earn its place; more context is not automatically better, because irrelevant text distracts and costs money.

    Why it matters: Most quality problems in retrieval and agent systems are context problems - the right fact was missing, buried, stale or drowned in noise - rather than model problems.

    Example: An agent's answers degrade after twenty steps. Inspecting the context shows old tool outputs filling half the window; summarising them and keeping only the current plan and key facts restores quality and halves the cost.

    Watch out: Fixing quality by adding more documents to every request. Check: log the assembled context for a sample of requests and look for missing, duplicated or irrelevant content.

    You can now: design what goes into each call; budget tokens across components; diagnose a failure by inspecting the assembled context.

    Choosing what goes in context

    Learn first: Context windows & knowledge cut-offs, Templates: data apart from instructions

    Select context by relevance to the current request, freshness, the user's permissions and the token budget. Give each component a budget - instructions, examples, retrieved passages, history - remove duplicates, and prefer short, precise snippets with their source over whole documents.

    Why it matters: The model can only use what it sees, and it uses focused context better than a pile of loosely related text.

    Example: For a billing question, the context includes the current billing policy section, the customer's last three invoices and a summary of the conversation - not the full policy manual or every past invoice.

    Watch out: Including documents the user is not allowed to see because they scored high on similarity. Check: for a sample of requests, confirm every context item is relevant, current and permitted for that user.

    You can now: set token budgets per context component; select context by relevance, freshness and permission; trim context without losing what matters.

    Long inputs: position & noise

    Learn first: Context windows & knowledge cut-offs

    Long context windows do not guarantee that a model uses all of their contents equally well. Research has found that models can under-use information placed in the middle of a long input, and irrelevant passages that look similar to the answer can mislead them. Placement matters: providers commonly advise putting long documents first and the question and instructions after them.

    Why it matters: "It fits in the window" is not the same as "the model will use it". Long-document features need testing for where the evidence sits.

    Example: A contract-review prompt finds clauses near the start and end reliably but misses one in the middle of a 90-page document; splitting the contract into sections and asking per section fixes it.

    Watch out: Assuming retrieval is unnecessary because the whole corpus fits. Check: test the same question with the key passage at the start, middle and end of the context and compare accuracy.

    You can now: explain position and distraction effects; structure long prompts sensibly; test whether long-context features really use their input.

    Summaries & compaction

    Learn first: Choosing what goes in context

    Long conversations and agent runs outgrow the context window and become slow and expensive. Compaction replaces older turns with a summary, or with structured state - decisions made, facts confirmed, open tasks - while keeping recent turns verbatim. Some providers and agent SDKs offer built-in context management, but what is kept is still your design decision.

    Why it matters: Without compaction, costs climb with every turn and the model eventually loses early instructions; with careless compaction, it loses a constraint the user stated once.

    Example: A support session keeps the last six messages, a running summary and a facts block with the order id and the customer's stated deadline, so the deadline survives even after the original message is summarised away.

    Watch out: Summaries that drop numbers, names or commitments. Check: test long sessions where a key fact appears only early, and assert the assistant still uses it after compaction.

    You can now: design a compaction strategy; keep critical facts as structured state; test that compaction preserves what matters.

    Prompt caching & stable prefixes

    Learn first: Usage, rate limits & budgets, Templates: data apart from instructions

    Providers can cache the processed form of a repeated prompt prefix - system instructions, tool definitions, long documents - so later requests that start with the same prefix are cheaper and faster. Some apply caching automatically; others let you mark cache breakpoints and choose how long entries live. Either way, caching only works if the stable content comes first and changes rarely.

    Why it matters: For assistants with long system prompts or document-heavy context, caching can cut input cost and time to first token substantially without changing quality.

    Example: Moving the per-user details from the top of the prompt to the end, after the shared instructions and tool definitions, lets most requests reuse the cached prefix; the usage field shows cached input tokens rising.

    Watch out: Putting timestamps, user names or random ids at the start of the prompt, which breaks the prefix every time. Check: read cached-token counts in the usage data before and after restructuring the prompt.

    You can now: explain how prompt caching works; order prompt content for cache hits; verify savings from usage data.

  6. 6. Ground answers in your own data

    Embeddings & search

    Learn first: Request anatomy: messages & roles

    An embedding model turns text (or images) into a vector of numbers so that items with similar meaning end up close together. Storing vectors for your documents and comparing them with the vector of a query gives semantic search - the retrieval half of most question-answering systems - and the same vectors support classification, clustering and deduplication.

    Why it matters: Retrieval quality caps answer quality: if search misses the right passage, no prompt can recover it. Understanding embeddings, similarity and indexes is how you debug a retrieval system.

    Example: A query about getting money back for a broken kettle retrieves the refund policy section even though it never says kettle, because their embeddings are close - while an exact product code still needs keyword search.

    Watch out: Treating embedding search as magic and never measuring what it returns. Check: build a small set of questions with known relevant passages and measure whether they appear in the top results.

    You can now: explain embeddings and semantic search; build and query a small vector index; combine semantic and keyword search where needed.

    Fresher: build one index by hand before using a framework. Experienced: focus on hybrid search and evaluating retrieval.

    What embeddings are

    Learn first: Core vocabulary for AI engineering

    An embedding is a fixed-length list of numbers produced by an embedding model to represent the meaning of a piece of text. Texts with similar meaning get vectors pointing in similar directions, so distance between vectors approximates difference in meaning. The numbers themselves are not interpretable; only comparisons between vectors from the same model mean anything.

    Why it matters: Embeddings let you search by meaning rather than exact words, which is what makes retrieval over natural-language questions work.

    Example: "How do I reset my password?" and "I can't log in, forgot my password" produce nearby vectors; "password policy for administrators" is further away even though it shares the word.

    Watch out: Comparing vectors from two different embedding models, which is meaningless. Check: store the model name and version with every vector and refuse comparisons across models.

    You can now: explain what an embedding represents; create embeddings with an API or a local model; compare texts by vector similarity.

    Choosing an embedding model

    Learn first: What embeddings are

    Embedding models differ in retrieval quality for your language and domain, vector size (which drives storage and speed), maximum input length, price and whether you can host them yourself. Public leaderboards are a useful shortlist, but the deciding test is retrieval accuracy on your own questions and documents.

    Why it matters: Changing embedding models later means re-embedding the whole corpus, so a measured choice up front saves time and money.

    Example: Two models score similarly on a public leaderboard, but on your Hindi and English support articles one finds the right passage in the top five for 88% of questions and the other for 71%.

    Watch out: Picking the largest vector size by default - it costs storage and speed and may not help. Check: compare two or three candidates on a labelled query set before indexing everything.

    You can now: shortlist embedding models; compare them on your own retrieval set; plan the cost of re-embedding.

    Vector stores & indexes

    Learn first: Similarity search & top-k, SQL & the data you query

    A vector store keeps embeddings with their text and metadata and answers nearest-neighbour queries, ideally combined with metadata filters. Options include pgvector inside PostgreSQL, dedicated databases such as Qdrant, Weaviate and Milvus, lightweight stores such as Chroma, libraries such as FAISS, and managed services. The concepts - collections, indexes, filters, upserts and deletes - transfer between them.

    Why it matters: The store you choose affects filtering, permissions, updates and operations far more than it affects similarity maths.

    Example: A team already running PostgreSQL stores documents, metadata and embeddings in one table with pgvector, filters by tenant and status in the same SQL query, and avoids running another database.

    Watch out: Choosing a store before knowing your filtering, update and scale needs. Check: confirm the store can filter by tenant and permission in the same query as the similarity search, and can delete a document cleanly.

    You can now: store and query vectors with metadata; use filters and upserts; choose a vector store from real requirements.

    Hybrid search: keywords + vectors

    Learn first: Similarity search & top-k

    Keyword search (such as BM25) excels at exact terms - product codes, names, error messages, rare words; vector search excels at paraphrase and meaning. Hybrid search runs both and merges the results, often with reciprocal rank fusion, which combines rankings without needing comparable scores.

    Why it matters: Pure vector search misses queries like "error E-4012" or an exact clause number, which users ask constantly. Hybrid search is a cheap, reliable improvement for most real corpora.

    Example: A query for "SKU 88-210 warranty" finds nothing useful by embeddings alone; keyword search finds the exact product page, and fusion puts it first while still surfacing the general warranty policy.

    Watch out: Adding hybrid search without measuring whether it helps your queries. Check: compare keyword, vector and hybrid hit rates on the same labelled set, including queries with codes and names.

    You can now: explain when keyword and vector search each win; implement a simple hybrid search; merge rankings with reciprocal rank fusion.

    Beyond RAG: classify, cluster, dedupe

    Optional on the Fresher track

    Learn first: Similarity search & top-k

    The same embeddings that power search support other tasks cheaply: assigning labels by similarity to labelled examples, clustering feedback into themes, finding near-duplicate tickets or documents, recommending similar items and flagging outliers that look unlike anything seen before.

    Why it matters: Many problems need no generative call at all; an embedding plus simple maths is faster, cheaper and easier to evaluate.

    Example: Instead of asking a model to read 20,000 reviews, you embed them, cluster them into themes, and ask a model to name each cluster from ten samples - a fraction of the cost.

    Watch out: Trusting clusters or nearest-neighbour labels without checking samples by hand. Check: review a random sample from each cluster or label and measure agreement before using the result.

    You can now: apply embeddings to classification, clustering and deduplication; compare the approach with a generative call on cost and accuracy; validate results by sampling.

    Retrieval-augmented generation (RAG)

    Learn first: Embeddings & search, Prompt structure that holds up

    RAG answers questions from your own data by retrieving relevant passages at request time and giving them to the model with instructions to answer from them and cite them. It keeps knowledge current without retraining, lets you control which sources are used, and makes answers checkable against their sources.

    Why it matters: Most "chat with our documents" and support features are RAG systems, and most of their failures come from ingestion and retrieval rather than from the model.

    Example: A policy assistant parses current policy PDFs, splits them by section, indexes them with version and region metadata, retrieves the top passages for each question, and answers with citations or says it cannot find the answer.

    Watch out: Judging the system only on final answers. Check: evaluate retrieval (was the right passage found?) separately from generation (was the answer faithful to it?).

    You can now: build a RAG pipeline end to end; locate failures by stage; make answers cite and stay within their sources.

    Fresher: build the simple pipeline first. Experienced: spend time on parsing, permissions, reranking and evaluation.

    The RAG pipeline end to end

    Learn first: Similarity search & top-k, Templates: data apart from instructions

    Ingestion: load documents, parse them to clean text, split into chunks, embed and store them with metadata. Query time: rewrite the question if needed, retrieve candidates with filters, rerank, assemble a context within budget, and generate an answer with citations - or abstain when the evidence is missing. Each stage can fail on its own and should be observable on its own.

    Why it matters: Seeing the pipeline as stages lets you find the broken one instead of tweaking the prompt.

    Example: An answer cites the wrong policy. The trace shows retrieval returned the right section second and an outdated one first; adding a version filter fixes it without touching the prompt.

    Watch out: A single end-to-end script with no logging of what was retrieved. Check: for any answer, you can see the chunks retrieved, their scores and the final context sent to the model.

    You can now: describe every stage of a RAG pipeline; build a minimal working pipeline; trace an answer back through each stage.

    Parsing PDFs, HTML & tables

    Learn first: The RAG pipeline end to end

    Retrieval can only find what parsing extracted. PDFs hide multi-column layouts, headers and footers, tables and scanned pages that need OCR; HTML carries navigation and boilerplate; tables lose meaning when flattened into lines of numbers. Good parsing keeps headings, section structure, tables and page numbers, and records where each piece came from.

    Why it matters: Garbled or missing text is invisible downstream: the system simply never retrieves the answer, and the model fills the gap.

    Example: A refund table parsed as a stream of numbers makes "electronics: 7 days" unretrievable; parsing it as a table, or converting each row to a sentence with its column headers, fixes the answer.

    Watch out: Never looking at the parsed output. Check: open the parsed text for a sample of documents, especially tables and scanned pages, and compare it with the original.

    You can now: extract clean text from PDFs and HTML; preserve structure, tables and page numbers; spot parsing failures before indexing.

    Chunking strategies

    Learn first: Parsing PDFs, HTML & tables

    Chunking splits documents into pieces small enough to retrieve precisely but large enough to stand alone. Common strategies are fixed-size chunks with overlap, recursive splitting on paragraphs and sentences, and structure-aware chunking by headings or sections; a parent-child approach retrieves small chunks but passes their larger parent section to the model.

    Why it matters: A chunk that separates a rule from its exception, or a heading from its content, produces confidently wrong answers.

    Example: "Refunds within 30 days" lands in one chunk and "except perishable items" in the next; switching to section-based chunks that keep the heading and the full rule together fixes the answer.

    Watch out: Choosing a chunk size by habit. Check: compare two or three chunking strategies on the same labelled questions and read the chunks that were retrieved for failures.

    You can now: apply several chunking strategies; keep context and headings with each chunk; choose a strategy from retrieval measurements.

    Metadata, filters & freshness

    Learn first: Chunking strategies, Vector stores & indexes

    Store metadata with every chunk: source, title, section, version or effective date, language, tenant and access permissions. Filters on that metadata narrow retrieval to what is current and allowed. Ingestion must also handle change - updating edited documents, deleting retired ones and re-indexing when parsing or embedding changes.

    Why it matters: Stale and unauthorised content are two of the most damaging RAG failures, and both are solved by metadata and indexing discipline, not by prompts.

    Example: Last year's refund policy keeps being retrieved; adding a status field and filtering to approved, current versions removes it, and a nightly job deletes chunks of retired documents.

    Watch out: An index that only ever grows. Check: change a source document, re-run ingestion and confirm the old version can no longer be retrieved.

    You can now: design chunk metadata; filter retrieval by freshness and scope; keep an index in sync with changing sources.

    Reranking

    Learn first: Similarity search & top-k

    A reranker re-scores retrieved candidates against the query with a more accurate but slower model - typically a cross-encoder that reads the query and passage together. A common pattern retrieves 30 to 50 candidates cheaply, reranks them and passes only the top few to the generator.

    Why it matters: Reranking often gives the biggest retrieval quality gain for the least effort, and it lets you send fewer, better passages to the model.

    Example: First-stage search puts the correct passage at position 9; the reranker moves it to position 1, so the answer improves and the prompt shrinks from ten passages to three.

    Watch out: Adding a reranker without checking the latency it adds. Check: measure hit rate at the final k and the added latency with and without reranking on the same queries.

    You can now: explain how cross-encoder reranking differs from embedding search; add a reranking step; measure its quality and latency effect.

    Query rewriting & routing

    Optional on the Fresher track

    Learn first: The RAG pipeline end to end

    User questions are often poor search queries: follow-ups that depend on earlier turns, several questions in one, or vague wording. Query rewriting turns them into standalone search queries; decomposition splits multi-part questions; routing sends a query to the right index, tool or prompt.

    Why it matters: In conversations, "what about for electronics?" retrieves nothing useful until it is rewritten as "refund window for electronics".

    Example: A rewriting step turns the follow-up into a standalone query using the last two turns; a router sends billing questions to the billing index and product questions to the catalogue search.

    Watch out: A rewrite that changes the meaning of the question. Check: log original and rewritten queries and include follow-up questions in your retrieval evaluation set.

    You can now: rewrite conversational queries for retrieval; route queries to sources; evaluate the effect of rewriting.

    Citations & grounded answers

    Learn first: The RAG pipeline end to end, Structured outputs & JSON Schema

    A grounded answer makes only claims supported by the supplied passages and says which passage supports each claim. Number or label passages in the prompt, require citations, instruct the model to say it does not know when the passages do not answer the question, and check citations - by code where possible and in your evaluations. Some APIs return citations natively.

    Why it matters: Citations let users and reviewers verify answers, and checking them catches the subtle failure where a real citation is attached to an unsupported claim.

    Example: An answer says refunds arrive in 24 hours and cites passage 3, which actually says 5 to 7 business days; a faithfulness check flags it, and the case joins the regression set.

    Watch out: Treating the presence of a citation as proof of support. Check: sample answers and confirm each cited passage really contains the claim; track the unsupported-claim rate over time.

    You can now: prompt for cited answers and abstention; verify citations; measure how often answers are unsupported.

    Permission-aware retrieval

    Security

    Learn first: Metadata, filters & freshness, Build a small API service

    In multi-user and multi-tenant systems, retrieval must only return documents the requesting user is allowed to see. Enforce this in the retrieval query - filtering by tenant and access lists derived from the authenticated session - or with separate indexes per tenant. The model must never be the component that decides what a user may see.

    Why it matters: A prompt that says "do not reveal other customers' documents" is not a control; once a private passage reaches the context, it can appear in an answer.

    Example: Every chunk carries a tenant id and allowed groups; the search API adds the signed-in user's tenant and groups as filters, and a test proves a query from tenant B can never return tenant A's private article.

    Watch out: Filtering after generation, or trusting a tenant id supplied in the request body. Check: run cross-tenant retrieval tests in CI and include caches and logs in the review.

    You can now: design access control for a retrieval index; derive filters from authenticated identity; test for cross-tenant leakage.

    RAG frameworks: LlamaIndex, LangChain & co

    Optional on the Fresher track · Optional on the Experienced track

    Learn first: The RAG pipeline end to end

    Frameworks such as LlamaIndex, LangChain and Haystack provide document loaders, splitters, retrievers, rerankers and pipelines, which speed up prototypes and offer many integrations. They also hide what is happening and change quickly between versions. Learn the pipeline by hand first, then adopt one framework if it saves real effort.

    Why it matters: Debugging retrieval requires seeing exactly what was loaded, split and retrieved; that is easier when you understand what the framework does underneath.

    Example: A team builds its first pipeline with plain code to learn the stages, then adopts a framework's loaders for twenty file formats while keeping its own chunking, filtering and evaluation.

    Watch out: Learning several frameworks instead of the concepts, and leaving versions unpinned. Check: you can explain what each framework component in your pipeline does and replace it with plain code if needed.

    You can now: evaluate whether a framework helps; use one for loading and orchestration; keep control of the parts that decide quality.

  7. 7. Measure quality before you ship

    Evaluation & testing

    Learn first: Structured outputs & JSON Schema, Testing with pytest

    Evaluation is how you know whether an AI feature works and whether a change made it better or worse. It combines a dataset of representative inputs, checks that score each output - deterministic code checks first, model-graded or human judgement where needed - and a process that runs those checks on every prompt, model or code change.

    Why it matters: Without evaluation every change is a guess, demos hide failures, and nobody can answer "is it good enough to ship?" with evidence. Evaluation is also what interviewers probe hardest.

    Example: A support assistant has 150 test conversations with expected facts, three code checks (schema, cited source exists, no forbidden promises) and a calibrated judge for tone; every pull request shows the scores beside the previous release.

    Watch out: A single average score that hides a broken segment. Check: report results per category, language and difficulty, and treat safety failures as blocking whatever the average.

    You can now: design an evaluation for a feature; combine code checks, judges and human review; use results to make release decisions.

    Fresher: start with code checks on a small dataset. Experienced: build regression gates and fair comparisons.

    Success criteria & what to measure

    Learn first: Hallucination & other failure modes

    Before measuring anything, write down what good means for the feature: the outcomes that matter (correct answer, faithful to sources, right format, safe, helpful tone), how each will be measured, the threshold that is good enough, and which failures block a release outright. Different features need different criteria - an extractor cares about field accuracy, a support bot about groundedness and escalation.

    Why it matters: Vague goals such as "make it smarter" cannot be tested or agreed on; specific criteria turn arguments into measurements.

    Example: For a refund assistant: at least 95% of answers grounded in the cited policy, zero unsupported refund approvals, valid JSON in 100% of responses, and p95 latency under four seconds.

    Watch out: Measuring what is easy, such as response length or friendliness, instead of the harm you care about. Check: for each criterion, name the user harm it prevents; drop or demote criteria that prevent none.

    You can now: turn a feature goal into measurable criteria; separate blocking from non-blocking metrics; set thresholds stakeholders can agree to.

    Building an eval dataset

    Learn first: Success criteria & what to measure

    An evaluation dataset is a versioned set of inputs with the expected output or the facts a good output must contain. Draw it from real usage where you can (with personal data removed), cover each category and language users bring, add edge cases and adversarial inputs, and keep it separate from the examples used to write prompts. Start small - twenty to fifty cases - and grow it from every failure found in testing or production.

    Why it matters: The dataset defines what "better" means. An unrepresentative set produces confident numbers about the wrong thing.

    Example: A triage set starts with 40 anonymised tickets across six categories and two languages; each production bug adds a case, and after two months it has 180 cases with a known history of what each one caught.

    Watch out: Testing only the happy path, and reusing evaluation cases as few-shot examples. Check: compare the category mix of the dataset with real traffic, and confirm no test case appears in any prompt.

    You can now: assemble and label an evaluation set; cover segments, edge cases and attacks; version the dataset and grow it from failures.

    Code-based checks & assertions

    Learn first: Building an eval dataset, Testing with pytest

    Many qualities can be checked with ordinary code: the output parses and matches the schema, enum values are allowed, numbers are within range or match the expected value within a tolerance, required facts or citations appear, cited passages were actually in the context, forbidden phrases or actions are absent. These checks are fast, free, deterministic and easy to trust.

    Why it matters: Code checks catch most regressions cheaply and give unambiguous results, so model-graded evaluation can be saved for genuinely subjective qualities.

    Example: For an extraction task, code checks compare each field with the labelled value, accept dates in any valid format by normalising them first, and fail any output whose total differs from the sum of its line items.

    Watch out: Exact string matching on free text, which fails correct answers phrased differently. Check: read a sample of failures from each check and confirm they are real failures, not artefacts of a brittle check.

    You can now: write deterministic graders for structured and semi-structured outputs; choose tolerant comparisons where needed; trust and debug your checks.

    Model-graded evals & their limits

    Learn first: Code-based checks & assertions

    A model can grade outputs against a rubric when quality is open-ended - helpfulness, tone, faithfulness of a summary. Judges work best with narrow, clearly defined, usually pass-or-fail criteria and the evidence needed to decide. They have known biases: favouring longer answers, the first option shown or text in their own style, and they can be swayed by instructions hidden in the answer they are grading.

    Why it matters: Judges scale human-like grading, but an uncalibrated judge gives you precise-looking numbers that may not match what people think.

    Example: You label 60 answers by hand for faithfulness, run the judge on the same answers and find 92% agreement; the disagreements reveal a rubric ambiguity, which you fix before trusting the judge on new data.

    Watch out: Using a judge whose agreement with human labels was never measured, and letting the graded text instruct the judge. Check: measure judge-human agreement on a labelled sample and re-check it whenever the rubric or judge model changes.

    You can now: write a rubric for a model grader; calibrate a judge against human labels; recognise and reduce judge bias and manipulation.

    Retrieval metrics: hit rate, recall & MRR

    Learn first: The RAG pipeline end to end, Building an eval dataset

    Retrieval is evaluated with questions whose relevant passages are known. Hit rate at k asks whether any relevant passage appears in the top k; recall at k asks what fraction of all relevant passages appear; precision at k asks what fraction of the top k are relevant; mean reciprocal rank rewards putting the first relevant passage near the top.

    Why it matters: Retrieval caps answer quality. Measuring it separately tells you whether to fix search or fix generation.

    Example: Ten questions, top-3 retrieval: the relevant passage appears for seven of them, so hit rate at 3 is 70%; if the first relevant result was at rank 1 for five questions and rank 2 for two, MRR is (5 x 1 + 2 x 0.5) / 10 = 0.6.

    Watch out: Reporting answer accuracy alone and guessing where failures came from. Check: for each failed answer, look up whether retrieval found the passage; tally failures by stage.

    You can now: label a retrieval test set; compute hit rate, recall, precision and MRR; use the numbers to choose chunking, k and reranking.

    Task success & end-to-end evals

    Learn first: Code-based checks & assertions

    For workflows and agents, the question is whether the task was completed correctly, not whether the transcript sounds right. Define the expected end state - the ticket updated with the right fields, the refund correctly refused, the file changed and tests passing - and check it in code, along with the steps taken, tool calls made, cost and any policy violations.

    Why it matters: Agents can produce convincing explanations of work they did not do, or succeed by an unsafe route. Checking outcomes and paths catches both.

    Example: An agent asked to cancel a duplicate order is graded on the order system's final state, on whether it confirmed with the user before cancelling, and on staying under ten tool calls.

    Watch out: Grading only the agent's final message. Check: every agent evaluation case asserts the real end state and at least one property of the path, such as no forbidden tool calls.

    You can now: define end states for tasks; evaluate agents on outcomes and paths; track steps, cost and violations per task.

    Regression tests in CI

    Learn first: Code-based checks & assertions, Prompt versioning & iteration

    A regression suite runs automatically when prompts, models, retrieval settings or code change, and blocks the change if blocking metrics fall below their thresholds. Keep a fast subset for every pull request and run the full set on a schedule or before release; pin model versions and settings so results are comparable; store results so trends are visible.

    Why it matters: Model behaviour changes in surprising places. Automatic gates catch the regression in review instead of in production.

    Example: A pull request that shortens the system prompt passes unit tests but fails the regression gate because grounded-answer rate falls from 96% to 89% on the policy set; the author fixes it before merging.

    Watch out: Gates so noisy that people learn to ignore them, and suites so expensive that nobody runs them. Check: rerun the suite twice on an unchanged system to measure noise, then set thresholds outside that noise.

    You can now: wire an evaluation suite into CI; define blocking thresholds; manage the cost and noise of automated evals.

    Comparing prompts & models fairly

    Learn first: Building an eval dataset

    A fair comparison runs every candidate on the same frozen dataset with the same settings, repeats runs where outputs vary, compares results case by case rather than only on averages, and asks whether the difference is larger than the noise. Small datasets produce large swings; a two-point difference on 30 cases is usually not evidence.

    Why it matters: Teams routinely ship "improvements" that were sampling noise, or regressions hidden by a favourable test set.

    Example: Prompt B beats prompt A by three points overall; case by case, B wins 14 cases and loses 9, and the losses are all in Hindi. You keep A for Hindi traffic and investigate B's failure before deciding.

    Watch out: Declaring a winner from one run on a handful of examples, and tuning on the same data you report. Check: report the number of cases, the wins and losses per case, and the variation across repeated runs.

    You can now: design a fair comparison; read paired results; decide whether a difference is real enough to act on.

    Error analysis & human review

    Learn first: Building an eval dataset

    Error analysis means reading real outputs and traces, writing down what went wrong in each failure, grouping failures into categories and counting them, so you fix the largest category first. Human review of a regular sample keeps the evaluation honest and finds failure types no metric was looking for.

    Why it matters: Aggregate scores tell you that something is wrong; reading failures tells you what to fix. It is the highest-value hour in most AI projects.

    Example: Reviewing 50 bad answers shows 22 caused by missing documents in the index, 15 by an outdated policy, 8 by formatting and 5 by genuine model errors - so the next week goes to ingestion, not prompt tweaks.

    Watch out: Jumping to a fix after reading three failures. Check: categorise at least 30 to 50 failures before choosing what to work on, and turn each category into evaluation cases.

    You can now: run a structured error analysis; build a failure taxonomy; prioritise fixes by measured impact.

    Eval tooling: promptfoo, DeepEval, Ragas & more

    Optional on the Fresher track · Optional on the Experienced track

    Learn first: Code-based checks & assertions, Model-graded evals & their limits

    Open-source tools package common evaluation work: test-case configuration, assertions, model graders, retrieval metrics, red-team scans and reports. Examples include promptfoo, DeepEval, Ragas and Inspect; observability platforms add dataset and experiment tracking. Tools come and go, so check that a project is actively maintained before you build on it. They save time, but the dataset and criteria are still yours to design.

    Why it matters: A tool can make evaluations repeatable and shareable across a team, as long as you understand what each metric computes.

    Example: A team keeps its own dataset and code checks, and uses an evaluation tool to run them across three models in parallel and produce a side-by-side report for the pull request.

    Watch out: Adopting a tool's built-in metrics without reading how they are calculated. Check: for any metric you report, you can explain its formula or prompt and have checked it on examples you labelled.

    You can now: run an evaluation suite with an open-source tool; interpret its built-in metrics critically; integrate it into CI.

  8. 8. Workflows first, agents when needed

    Agents & workflows

    Learn first: Tool (function) calling, Timeouts, retries & backoff

    An agentic system lets a model take actions through tools, either as a step in a workflow your code controls or as an agent that decides its own next step in a loop. The engineering challenge is control: state that survives failures, actions that are safe to retry, limits on steps and spending, and human approval where mistakes are costly.

    Why it matters: Agents unlock multi-step work, but an unbounded loop with powerful tools is an outage, a security incident or a large bill waiting to happen. Teams hire for people who can make them predictable.

    Example: A refund assistant is built as a workflow: the model classifies the request and drafts a reply, but eligibility, amount and execution are deterministic code, and refunds above a threshold need a person to approve.

    Watch out: Starting with a fully autonomous agent because it demos well. Check: for each step, ask whether code could decide it; give the model only the decisions code cannot make.

    You can now: choose between workflows and agents; build a bounded tool-using loop; add checkpoints, idempotency and approvals.

    Fresher: build one workflow and one small agent loop by hand. Experienced: focus on durability, bounds and approvals, then pick a framework.

    Deterministic workflows vs agent loops

    Learn first: Tool (function) calling

    In a workflow, code decides the sequence of steps and the model fills in specific parts - classify, extract, draft. In an agent, the model decides which tool to use next, observes the result and repeats until it judges the task done. Workflows are predictable, cheaper and easier to test; agents handle open-ended tasks where the steps cannot be known in advance.

    Why it matters: Most production systems are mostly workflow with a little agency. Choosing the simplest structure that works avoids most reliability problems.

    Example: "Summarise each new ticket and route it" is a two-step workflow. "Investigate why this customer's last three orders failed and propose a fix" may justify an agent with read-only tools and a step limit.

    Watch out: Using an agent to repeat a fixed sequence that code could run. Check: write the steps of your task down; if you can, implement them as a workflow first and add agency only where a step genuinely needs judgement.

    You can now: explain the difference between workflows and agents; choose the right structure for a task; justify the choice in a design review.

    Workflow patterns: chain, route, parallelise

    Learn first: Deterministic workflows vs agent loops

    A few composable patterns cover most workflows: prompt chaining (one step's output feeds the next, with checks between), routing (classify the input, then send it to a specialised path), parallelisation (run independent sub-tasks at once, or run the same task several times and vote), orchestrator-workers (one call plans sub-tasks that others execute) and evaluator-optimiser (one call drafts, another critiques against criteria, repeat with a limit).

    Why it matters: Recognising the pattern a task needs gives you a tested design and a clear place for checks, instead of one giant prompt.

    Example: A document assistant routes by type, extracts fields from invoices with one prompt and contracts with another, runs three clause checks in parallel and validates the combined result in code.

    Watch out: Chains with no validation between steps, so one bad output spoils everything after it. Check: add a schema or rule check after every step and log where chains fail.

    You can now: apply chaining, routing, parallelisation, orchestration and evaluator loops; put validation between steps; pick a pattern for a given task.

    The agent loop: plan, act, observe

    Learn first: Tool (function) calling

    An agent loop gives a model a goal, instructions and tools. Each turn, the model either answers or requests a tool call; your code checks and runs the tool and appends the result to the context; the loop continues until the model finishes or a limit is hit. Reasoning-and-acting styles such as ReAct interleave short reasoning with each action.

    Why it matters: Building the loop yourself once makes every agent framework understandable and shows where the controls go: tool validation, limits, state and logging.

    Example: A research agent with search and read tools answers a question in five turns; the trace shows each tool call, its result and the model's decision, so a wrong answer can be traced to a misleading search result.

    Watch out: Loops with no stop condition other than the model deciding it is done. Check: every loop has a maximum number of steps, a deadline and a cost budget, and logs each turn.

    You can now: implement a tool-using agent loop from scratch; trace an agent's decisions; place validation and limits correctly.

    State & memory

    Learn first: The agent loop: plan, act, observe, Summaries & compaction

    Agents need several kinds of memory: the working context of the current task, structured state such as the plan, decisions and collected facts, and sometimes long-term memory - facts about a user or past tasks stored outside the model and retrieved when relevant. Each needs a clear scope (this task, this user, this tenant), a size limit and rules for what is kept and forgotten.

    Why it matters: State that is never written down gets lost when context is trimmed; state shared too widely leaks between users.

    Example: A support agent keeps the current order id and the customer's stated constraints in a structured state object, stores long-term preferences per user only with consent, and never reads another user's memory.

    Watch out: Treating the whole chat transcript as memory, and storing personal data in memory without a retention rule. Check: inspect the state after a long task and confirm it is accurate, minimal and scoped to the right user.

    You can now: design task state and long-term memory; scope memory to users and tenants; decide what to remember and what to drop.

    Checkpoints & resumable runs

    Learn first: The agent loop: plan, act, observe

    A long-running workflow or agent should save its state after each completed step - a checkpoint - so a crash, deploy, timeout or failed tool call resumes from the last good step instead of starting over. Durable execution engines and agent frameworks provide checkpointing, but the principle is simple: persist inputs, outputs and decisions as you go, and make each step safe to re-run.

    Why it matters: Restarting a 40-step task from scratch wastes money and can repeat actions that already happened. Checkpoints also make runs replayable for debugging.

    Example: A document-processing run has finished 300 of 500 files when the provider returns errors for ten minutes; the job pauses, then resumes at file 301 using the saved checkpoint, and nothing is processed twice.

    Watch out: Saving checkpoints but not making steps idempotent, so a resumed step repeats a side effect. Check: kill a run midway in a test and confirm it resumes correctly without duplicate actions.

    You can now: add checkpoints to a multi-step run; resume after failures; replay a run for debugging.

    Idempotent actions & safe retries

    Learn first: Timeouts, retries & backoff

    An idempotent action has the same effect whether it runs once or several times. For actions with side effects - payments, refunds, emails, record updates - send an idempotency key the receiving service uses to recognise repeats, check current state before acting, and record the outcome. When a call times out, the outcome is unknown: look it up before trying again.

    Why it matters: Retries are everywhere in AI systems - in clients, queues and agent loops. Without idempotency, a timeout can turn into a double refund or a duplicate message.

    Example: A refund request times out. Instead of retrying with a new request, the workflow queries the payment provider with the original idempotency key, finds the refund succeeded, and reports success without issuing another.

    Watch out: Generating a new idempotency key on each retry, which defeats the purpose. Check: test the timeout-then-retry path and assert exactly one side effect happened.

    You can now: design idempotent tool actions; handle unknown outcomes after timeouts; prove in tests that retries cannot duplicate side effects.

    Step, time & cost limits

    Learn first: The agent loop: plan, act, observe, Usage, rate limits & budgets

    Every agent run needs hard limits enforced by code: a maximum number of steps or tool calls, a wall-clock deadline, a token or cost budget, and detection of loops such as the same tool called with the same arguments repeatedly. When a limit is hit, stop cleanly, keep any partial result, and tell the user or calling system what happened.

    Why it matters: Models can retry a failing tool forever, wander off task or get stuck in cycles. Limits turn those failures into bounded, reportable events.

    Example: An agent calls a failing search tool four times with identical arguments; loop detection stops it, the run returns its partial findings with a note that search was unavailable, and an alert fires.

    Watch out: Relying on the model to notice it is stuck, and reporting success when the run was cut off. Check: test each limit and confirm the run ends with an explicit, honest status.

    You can now: enforce step, time and cost limits; detect loops; end runs gracefully with an honest status.

    Human approval for risky actions

    Learn first: The agent loop: plan, act, observe

    For actions that are expensive, irreversible or affect other people - payments, deletions, external messages, production changes - an agent should propose and a person should approve. Show the approver the exact action and parameters, record who approved what, ask again if anything changes after approval, and define what happens if nobody responds.

    Why it matters: Approval gates let you ship useful agents before you can trust them fully, and they create the evidence needed to automate more later.

    Example: An agent proposes a refund of 1,200 rupees; the approver sees the order, the amount and the policy rule; the agent then recalculates the amount as 1,450, so the earlier approval is void and a new one is requested.

    Watch out: Approval screens that show a vague summary instead of the actual action, and approvals that carry over after parameters change. Check: test that any change to a proposed action invalidates its approval.

    You can now: decide which actions need approval; design an approval step that shows exact parameters; handle changed proposals and timeouts.

    Agent frameworks: pick one

    Optional on the Fresher track

    Learn first: The agent loop: plan, act, observe, Checkpoints & resumable runs

    Agent frameworks and SDKs - for example LangGraph, the OpenAI Agents SDK, the Claude Agent SDK, Google's Agent Development Kit and PydanticAI - provide tool definitions, loops, state, tracing, handoffs and sometimes durable execution. They differ in style and in how much they hide; the concepts underneath are the ones on this roadmap.

    Why it matters: A framework can save weeks on tracing, persistence and integrations, but learning several of them teaches less than mastering the concepts and one tool.

    Example: After building a small loop by hand, a team picks one framework because it offers checkpointing and human-in-the-loop interrupts they need, and writes a short decision note listing what they gave up.

    Watch out: Choosing a framework from social media popularity, and not pinning its version. Check: pick a framework only after listing the features you need, and confirm you can see every prompt and tool call it makes.

    You can now: compare frameworks against your requirements; build an agent with one of them; keep the ability to debug what it does.

    Multi-agent systems

    Optional on the Fresher track · Optional on the Experienced track

    Learn first: Workflow patterns: chain, route, parallelise, Step, time & cost limits

    Multi-agent systems split work across several model-driven agents - a planner and workers, specialists with different tools, or reviewers that check each other. They can help with broad parallel research or with separating duties, but they multiply cost, latency and the number of ways a run can fail.

    Why it matters: Many problems marketed as multi-agent are solved better by one agent with good tools or a simple workflow. Knowing the trade-off keeps systems simple.

    Example: A research task that needs twenty sources gathered in parallel uses an orchestrator and parallel workers, each with a strict budget, while a customer support flow stays single-agent.

    Watch out: Adding agents to fix a problem that is really poor tools or poor context. Check: compare a single-agent baseline with the multi-agent design on the same evaluation set, including cost.

    You can now: describe common multi-agent designs; judge when they are worth their cost; evaluate them against a simpler baseline.

  9. 9. Connect models to tools and data

    MCP & tool integrations

    Learn first: Tool (function) calling

    The Model Context Protocol (MCP) is an open standard for connecting AI applications to tools and data. A server exposes capabilities once, and any compatible host - a chat app, an IDE, an agent framework - can use them through a client. The specification is versioned by date; the latest revision, 2026-07-28, made the protocol stateless, and SDKs handle compatibility with earlier revisions.

    Why it matters: MCP replaces one-off integrations with a common interface, so the same database, ticketing or search server works across many AI applications - and its security model becomes part of your system's security.

    Example: A team wraps its internal knowledge base and ticket system as two MCP servers; engineers use them from their IDE assistant, and the support agent uses the same servers with narrower permissions.

    Watch out: Installing third-party servers without reviewing what their tools can do. Check: for every server you connect, list its tools, the data and actions they reach, and the credentials they use.

    You can now: explain MCP's purpose and architecture; build and connect a server; secure servers and the credentials they hold.

    Fresher: build a local server first. Experienced: focus on remote transport, authorization and security.

    Hosts, clients & servers

    Learn first: Tool (function) calling

    In MCP, the host is the AI application the user interacts with; it creates one client connector per server it uses; each server exposes capabilities such as tools and resources. Messages are JSON-RPC. Under the 2026-07-28 revision, every request carries its protocol version and client capabilities, servers advertise theirs through a discovery call, and there is no connection handshake or protocol-level session. Earlier revisions began with an initialize handshake, which current SDKs can still negotiate.

    Why it matters: Knowing which component does what tells you where to put each control: the host decides what the model may call and asks the user, the server enforces permissions on its own data.

    Example: An IDE (host) connects to a Git server and a database server through two clients; the model asks to run a query, the host asks the user, and the database server checks the query against its own read-only permissions.

    Watch out: Assuming the model talks to servers directly - it only proposes calls that the host makes. Check: draw the host, clients and servers for one of your setups and mark where consent and authorization happen.

    You can now: describe host, client and server roles; read an MCP message exchange; place consent and permission checks correctly.

    Tools, resources & prompts

    Learn first: Hosts, clients & servers

    Servers offer three main primitives. Tools are actions the model can request, each with a name, description and input schema. Resources are data the application can read, identified by URIs, such as files, records or documents. Prompts are reusable templates that a user can select. Clients can also support elicitation, which lets a server ask the user for missing input. The 2026-07-28 revision deprecates two older client features - roots and sampling - in favour of passing files through tool arguments or resources and calling model APIs directly, and carries server-to-client requests as multi-round-trip requests.

    Why it matters: Choosing the right primitive keeps control where it belongs: model-driven actions as tools, application-controlled context as resources, user-driven workflows as prompts.

    Example: A documentation server exposes search_docs as a tool, each page as a resource the app can attach, and a summarise-changes prompt the user can pick from a menu.

    Watch out: Exposing everything as tools, which hands the model decisions the application or user should make. Check: for each capability, ask who should trigger it - model, application or user - and pick the primitive accordingly.

    You can now: explain tools, resources and prompts; design a server's capabilities with the right primitives; describe the client-side features.

    Transports: stdio & Streamable HTTP

    Learn first: Hosts, clients & servers, HTTP, REST & JSON

    MCP defines two standard transports. With stdio, the client launches the server as a local subprocess and they exchange newline-delimited JSON-RPC over standard input and output. With Streamable HTTP, the server exposes one HTTP endpoint and every message is sent as an HTTP POST; the server replies with a JSON object or a server-sent-events stream scoped to that request. The older HTTP-with-SSE transport was replaced by Streamable HTTP in 2025.

    Why it matters: The transport decides deployment and security: stdio suits local tools running with the user's own credentials; Streamable HTTP suits shared, remote servers that need authentication.

    Example: A file-search server runs locally over stdio for one developer; the company's ticketing server runs remotely over Streamable HTTP behind authentication for the whole team.

    Watch out: Exposing an HTTP server without checking request origins - the specification requires servers to validate the Origin header to prevent DNS-rebinding attacks - or binding a local server to every network interface. Check: confirm origin validation, authentication and the bound address before running any HTTP server.

    You can now: choose stdio or Streamable HTTP; run a server over each; apply the transport's security requirements.

    Authorization for remote servers

    Learn first: Transports: stdio & Streamable HTTP

    For HTTP-based transports, MCP authorization builds on OAuth 2.1. A protected MCP server acts as an OAuth resource server and must publish protected resource metadata that tells clients which authorization server to use; clients use PKCE and resource indicators so that tokens are issued for that specific server. Servers must accept only tokens issued for themselves and must never pass a client's token through to another service. Local stdio servers take credentials from their environment instead.

    Why it matters: Remote servers often reach sensitive systems. Getting token audience and scopes right is the difference between a connector and an open door.

    Example: A CRM server returns 401 with a pointer to its metadata; the client discovers the authorization server, the user signs in and grants read-only scopes, and the server rejects a token that was issued for a different service.

    Watch out: Forwarding the user's token to downstream APIs, and asking for broad scopes up front. Check: test that the server rejects tokens with the wrong audience and that each tool works with the minimum scope.

    You can now: explain MCP's OAuth-based authorization flow; configure a protected server; enforce token audience and least-privilege scopes.

    Build an MCP server

    Learn first: Tools, resources & prompts, Type hints & Pydantic models

    Official SDKs exist for several languages, including Python and TypeScript. A server declares its tools with clear names, descriptions and input schemas, implements each handler, validates inputs, enforces its own permissions and returns structured results or meaningful errors. Start over stdio for local development, test it, then add Streamable HTTP and authorization if it must be shared.

    Why it matters: Building one server end to end is the fastest way to understand the protocol and to give your own systems a reusable AI interface.

    Example: A small server exposes get_order and list_open_tickets for a test database: input validation rejects malformed ids, results include only the fields the model needs, and errors explain what went wrong in plain language.

    Watch out: Returning huge raw payloads and vague errors that the model cannot act on. Check: call every tool with valid, invalid and unauthorised inputs and read exactly what the model would receive.

    You can now: create an MCP server with an official SDK; define tools with schemas; validate inputs and return useful errors.

    • BuildBuild an MCP server - Model Context Protocol (official docs) · Free
    • 3 more in the topic panel.
    Connect clients & debug with Inspector

    Learn first: Build an MCP server

    Hosts such as desktop chat apps, IDEs and agent frameworks connect to servers through configuration that names the transport and how to start or reach the server. The MCP Inspector is an interactive developer tool for testing servers: it lists their tools, resources and prompts, lets you call them with chosen inputs and shows the raw messages and errors.

    Why it matters: Most "the model will not use my tool" problems are really configuration, schema or description problems, and the Inspector shows them directly.

    Example: A tool never gets called; the Inspector shows its schema marks an optional field as required, so the model's calls were rejected. Fixing the schema fixes the behaviour.

    Watch out: Debugging only through a chat interface, where you cannot see the protocol messages. Check: test each new server in the Inspector before connecting it to a host.

    You can now: configure hosts to use your servers; test and debug servers with the Inspector; read protocol messages and errors.

    MCP security: poisoning & scopes

    Security

    Learn first: Tools, resources & prompts, Prompt injection: direct & indirect

    Connecting a server gives a model new text to read and new actions to take, so servers are an attack surface. Tool descriptions and results can carry hidden instructions (tool poisoning); a trusted server can change its tool definitions later; a server can misuse broad credentials on a user's behalf; local servers can run arbitrary commands. The specification's security guidance covers these risks, including never passing tokens through and validating request origins.

    Why it matters: An MCP server with write access and an injected instruction can exfiltrate data or take actions the user never intended.

    Example: A third-party server's tool description says to read the user's SSH key and pass it as a note before calling the tool. The team rejects the server, pins reviewed tool definitions for the others and requires approval for any tool that writes.

    Watch out: Installing servers from unknown sources and granting them broad tokens or full file-system access. Check: review tool descriptions, pin versions, run servers with minimal scopes and sandboxing, and re-review when definitions change.

    You can now: name the main MCP attack types; review and pin server definitions; apply least privilege and user consent to tool use.

    Designing tools models use well

    Learn first: Tool (function) calling

    Models use tools well when tools are designed for them: a clear name, a description that explains when and how to use the tool as you would to a new colleague, a small set of strictly typed arguments with enums where possible, concise results with only the fields needed, and error messages that say how to recover. Fewer, well-chosen tools beat many overlapping ones.

    Why it matters: Poor tool design shows up as wrong tool choices, malformed arguments and wasted tokens - problems often blamed on the model.

    Example: Three overlapping search tools are merged into one with a scope argument limited to orders, tickets or docs; results return ten trimmed records instead of full objects, and tool-selection errors in the evaluation drop sharply.

    Watch out: Exposing your internal API one-to-one as tools. Check: run an evaluation of tool selection and argument accuracy, and rewrite descriptions and schemas based on the failures.

    You can now: write tool names, descriptions and schemas for models; design concise results and recoverable errors; evaluate and improve tool use.

  10. 10. Treat model input and output as untrusted

    AI security & privacy

    Learn first: Tool (function) calling, The RAG pipeline end to end

    LLM features create new attack paths: instructions hidden in user input, documents or tool results; model output that is executed or rendered without checks; agents with more permission than they need; secrets and personal data flowing into prompts and logs; and one user's data reaching another. The defences are mostly classic security - least privilege, validation, isolation, auditing - applied to a component that cannot be trusted to follow rules.

    Why it matters: A model can be talked into anything, so every control that matters must hold even when the model is fooled. Security review is now a standard gate for AI features.

    Example: A support assistant reads web pages and can create tickets. An injected page tells it to email the customer database; the attack fails because the ticket tool cannot send email, the assistant has no database access, and outbound actions need approval.

    Watch out: Relying on prompts such as "never reveal secrets" as the control. Check: for each risky outcome, name the control outside the model that prevents it.

    You can now: threat-model an LLM feature; apply least privilege and isolation; test your own system against common attacks.

    Fresher: learn prompt injection, privacy and secrets first. Experienced: focus on agency, tenancy and red-teaming.

    Prompt injection: direct & indirect

    Learn first: System prompts & instruction priority

    Prompt injection is text that tries to change a model's instructions. Direct injection comes from the user ("ignore your rules and..."); indirect injection hides instructions in content the model processes - a web page, an email, a PDF, a tool result or a retrieved document. Models cannot reliably tell instructions from data, so injection is mitigated rather than solved.

    Why it matters: Injection is the signature attack on LLM applications, and indirect injection grows more dangerous as agents read more external content and gain more tools.

    Example: A help article contains a hidden line telling the assistant to reveal its API key. The assistant has no key in its context, treats the article as data, and the output filter blocks anything that looks like a credential - so the attack achieves nothing.

    Watch out: Defences that are only prompt wording. Check: combine delimiting untrusted content, minimal privileges, approval for actions, output checks and an injection test set, and run that test set on every change.

    You can now: explain direct and indirect injection; design layered defences; build and run an injection regression test set.

    Handling untrusted model output

    Learn first: Structured outputs & JSON Schema

    Treat model output like user input: validate it before passing it to other systems, encode it before rendering it in a browser, never execute it as code, SQL or shell commands without strict constraints and sandboxing, and check URLs and file paths it produces. An attacker who controls part of the input may control part of the output.

    Why it matters: Unsafe output handling turns a prompt injection into cross-site scripting, data exfiltration through generated links, or code execution on your servers.

    Example: A chatbot renders markdown; an injected document makes it output an image link whose URL contains the user's private data. Rendering only allow-listed domains and stripping remote images closes the channel.

    Watch out: Rendering raw HTML or markdown from the model, and running generated code outside a sandbox. Check: pass a set of malicious outputs - script tags, data-carrying URLs, shell commands - through your rendering and execution paths in tests.

    You can now: validate and encode model output; sandbox or constrain generated code; close data-exfiltration channels in rendering.

    Least privilege for tools & agents

    Learn first: Tool (function) calling

    Give a model only the tools, permissions and data it needs for the task, scoped to the current user. Prefer read-only tools; separate read and write; limit what each write tool can change; require approval for consequential actions; and enforce all of it in the tools themselves, not in the prompt.

    Why it matters: Excessive agency is what turns a model mistake or an injected instruction into real damage. The less a compromised agent can do, the smaller the incident.

    Example: An email assistant needs to draft replies, so it gets read access to the current thread and a create-draft tool - not send, not delete, not access to other mailboxes.

    Watch out: Giving an agent an admin token "to make development easier" and forgetting to narrow it. Check: list every tool and credential an agent holds and justify each against a specific task; remove the rest.

    You can now: scope tools and credentials to a task; separate read and write capabilities; enforce limits in code and require approvals where needed.

    Secrets & credentials

    Learn first: Config, env vars & secrets

    API keys, database passwords and tokens belong in a secrets manager or environment configuration, never in prompts, code, logs, evaluation data or error messages. Tools that need credentials hold them server-side and use them on the model's behalf; the model never sees them. Keys are scoped narrowly, rotated regularly and revoked immediately if exposed.

    Why it matters: Anything in a model's context can end up in its output, and anything in logs can end up in a support ticket or a vendor's system.

    Example: A debugging log line includes the full request headers, exposing the provider key; the fix redacts authorization headers in the logging layer, rotates the key and adds a test that fails if a key pattern appears in logs.

    Watch out: Putting credentials in a system prompt so the model can "use" them. Check: search prompts, logs and traces for key patterns automatically, and confirm every key has an owner and a rotation date.

    You can now: keep credentials out of model context and logs; let tools use secrets safely; respond to a leaked key.

    Privacy, PII & data retention

    Learn first: Request anatomy: messages & roles

    Send a model only the personal data a task needs, redact or pseudonymise the rest, know where requests are processed and how long the provider and your own logs keep them, and give users notice and choices as the law requires (for example GDPR in Europe or the DPDP Act in India). Provider terms and data-retention options differ, so read them for each platform you use.

    Why it matters: Prompts, traces and evaluation sets quietly accumulate personal data. Privacy failures are legal, financial and trust failures.

    Example: A ticket summariser receives the issue text and order id but not the customer's address or phone number; traces store a redacted copy for 30 days, and evaluation data is built from anonymised tickets.

    Watch out: Copying production conversations into evaluation sets or demos without redaction. Check: map where personal data flows - prompt, provider, logs, traces, datasets - and set a retention period for each.

    You can now: apply data minimisation to prompts; redact personal data; document data flows and retention for an AI feature.

    Tenant isolation & access control

    Learn first: Permission-aware retrieval

    In multi-user and multi-tenant products, every part of an AI feature must respect who the user is: retrieval filters, tool permissions, conversation history, caches, memory, logs and evaluation data. Identity comes from the authenticated session on the server, never from the prompt or the model's output.

    Why it matters: Cross-user leakage in AI features often comes from places nobody thought of - a shared response cache, a conversation store keyed by session alone, a vector index without tenant filters.

    Example: Two customers ask about their order status a second apart; a cache keyed only on the question text returns the first customer's answer to the second. Keying the cache by tenant and user, and testing it, fixes the leak.

    Watch out: Trusting a user id or role stated in a message, or supplied by the model. Check: write tests where user B tries to reach user A's data through every path - retrieval, tools, history, cache - and confirm each fails.

    You can now: trace identity through an AI feature; isolate data by tenant and user; test for cross-user leakage.

    Guardrails & moderation

    Learn first: Prompt injection: direct & indirect

    Guardrails are checks before and after the model: input checks for abuse, out-of-scope requests or injection patterns; output checks for policy violations, personal data, unsupported claims or unsafe actions; and topic or format constraints. They can be rules, classifiers, moderation APIs or a second model, and each must be tested for both misses and false alarms.

    Why it matters: Guardrails catch a share of failures the main prompt does not, but an over-strict guardrail blocks legitimate users and an untested one gives false confidence.

    Example: A refund assistant's output check blocks any reply that promises a refund the eligibility service did not approve; a test set includes legitimate refund questions to make sure the check does not block them.

    Watch out: Measuring only what a guardrail blocks and never what it wrongly blocks. Check: test each guardrail on harmful and harmless boundary cases and track both error rates.

    You can now: design input and output checks; choose between rules, classifiers and model checks; measure misses and false positives.

    Red-teaming your own app

    Learn first: Prompt injection: direct & indirect, Regression tests in CI

    Red-teaming means attacking your own system before others do: prompt injection through every input channel, jailbreak attempts, data-exfiltration tricks, attempts to reach other users' data, abuse of tools and attempts to run up costs. Open-source scanners and frameworks automate known attack families; manual testing finds what is specific to your product.

    Why it matters: Each new tool, data source or integration creates new attack paths, and the cheapest time to find them is before release.

    Example: Before launch, a team runs an automated injection scan, then spends a day manually attacking the document upload path and finds that instructions in PDF metadata reach the prompt - fixed by stripping metadata at parsing.

    Watch out: Running a scanner once and calling the system secure. Check: turn every successful attack into a regression test and re-run the suite whenever prompts, tools or models change.

    You can now: plan a red-team exercise; use automated scanners alongside manual testing; convert findings into regression tests.

    Bias, fairness & harm

    Learn first: Building an eval dataset

    Models reflect patterns in their training data, which can show up as different quality or treatment for different groups, stereotyped content, or poorer performance in some languages and dialects. Responsible features measure performance per group where it matters, avoid using models for decisions they cannot justify, and keep humans responsible for consequential outcomes.

    Why it matters: An assistant that works well on average but fails one language group or treats applicants differently by name causes real harm and real legal risk.

    Example: A resume screening assistant is evaluated on matched profiles that differ only in name and gender; scores differ, so the team removes the scoring feature and keeps the model to summarising, with decisions made by people.

    Watch out: Reporting only overall accuracy for a product used by diverse users. Check: break evaluation results down by language and relevant groups, and review decisions that affect people's opportunities or money.

    You can now: identify where bias can enter an AI feature; measure performance by group; limit model involvement in high-stakes decisions.

  11. 11. Beyond text: images, documents and voice

    Multimodal & voice

    Learn first: Request anatomy: messages & roles

    Many models now accept images and documents as input, and speech models turn audio into text and text into natural-sounding voice. This opens up document extraction from scans and photos, visual question answering, voice assistants and pipelines that move between formats. Each modality adds its own costs, latency and failure modes.

    Why it matters: A large share of real business data is scanned documents, screenshots, recordings and calls. Multimodal skills extend what you can automate.

    Example: A claims assistant reads a photo of a damaged parcel and the scanned receipt, extracts the order number and amount into a validated schema, and routes the claim - with a person checking any claim above a threshold.

    Watch out: Assuming vision and speech outputs are as reliable as typed text. Check: build an evaluation set of real images and recordings, including poor-quality ones, before trusting the pipeline.

    You can now: send images and documents to a model; transcribe and synthesise speech; design and evaluate a multimodal pipeline.

    Fresher: start with image and document understanding. Experienced: look at realtime voice and pipeline design.

    Image & document understanding

    Learn first: Request anatomy: messages & roles, Structured outputs & JSON Schema

    Vision-capable models can describe images, read text in screenshots and photos, interpret charts and extract fields from scanned documents, and several APIs accept PDFs directly. Accuracy drops with small or blurry text, dense tables and handwriting, and images consume input tokens in proportion to their size and detail settings.

    Why it matters: Document extraction from scans and photos is one of the most common business uses of AI, and it needs validation because misread numbers look plausible.

    Example: An invoice extractor returns line items and a total from a phone photo; code checks that the line items add up to the total and sends mismatches to a person instead of the accounting system.

    Watch out: Trusting extracted numbers without a cross-check. Check: validate extracted values with arithmetic, formats and lookups, and measure field accuracy on real, imperfect images.

    You can now: send images and PDFs to vision models; extract structured fields from documents; validate and route uncertain extractions.

    • OfficialVision - Anthropic (Claude Platform Docs) · Free
    • OfficialImages and vision - OpenAI · Free
    • 3 more in the topic panel.
    Speech-to-text & text-to-speech

    Learn first: Request anatomy: messages & roles

    Speech-to-text models transcribe audio, often with timestamps, language detection and sometimes speaker separation; open-source models such as Whisper run locally, and providers offer hosted transcription. Text-to-speech turns text into natural voices with controllable style. Transcription quality is measured with word error rate and depends on accents, noise, domain terms and language.

    Why it matters: Meetings, calls and voice notes are rich data, and voice is the natural interface for many users - but transcription errors flow into everything downstream.

    Example: A call-summary pipeline transcribes calls, corrects product names with a custom vocabulary list, summarises them and links each action item back to its timestamp so a person can check it.

    Watch out: Ignoring consent and disclosure rules for recording and for synthetic voices. Check: measure word error rate on your own recordings, including accented and noisy audio, and confirm consent is captured.

    You can now: transcribe audio with timestamps; generate speech; evaluate transcription quality on your own data.

    Realtime voice agents

    Optional on the Fresher track

    Learn first: Speech-to-text & text-to-speech, Streaming responses

    A voice agent either chains speech-to-text, a language model and text-to-speech, or uses a speech-to-speech realtime model, usually over WebRTC or WebSockets. The hard parts are conversational: detecting when the user has finished speaking, letting them interrupt, keeping the delay before the reply short and handling tools mid-conversation.

    Why it matters: In voice, a pause of a second and a half feels broken. Latency is a design budget split across every stage, not an afterthought.

    Example: A booking agent's reply delay is measured per stage - end-of-speech detection, transcription, time to the model's first sentence, time to first audio - and streaming the first sentence to speech synthesis cuts the perceived delay by more than a third.

    Watch out: Measuring only total response time and waiting for the full model reply before speaking. Check: log the time spent in each stage for real calls and set a budget for each.

    You can now: describe cascaded and speech-to-speech voice architectures; budget latency across stages; handle turn-taking and interruptions.

    Multimodal pipelines

    Learn first: Image & document understanding, Speech-to-text & text-to-speech

    Real features chain modalities: audio to transcript to summary to tasks; photo to extracted fields to database record; screen recording to steps to documentation. Each stage has its own model, error rate and cost, and errors compound, so each hand-off needs validation and the pipeline needs an end-to-end evaluation.

    Why it matters: A pipeline where every stage is 95% accurate can still be wrong a quarter of the time end to end. Designing checks between stages keeps quality acceptable.

    Example: Meeting recordings become action items: transcription with timestamps, a structured extraction of decisions and owners, a check that every owner is a meeting participant, and a link from each item to its timestamp for review.

    Watch out: Evaluating each stage separately and never the whole chain. Check: run complete recordings through the pipeline and score the final output against a human-written reference.

    You can now: design a multi-stage multimodal pipeline; validate between stages; evaluate end-to-end quality and cost.

    Image generation & provenance

    Optional on the Fresher track · Optional on the Experienced track

    Learn first: Request anatomy: messages & roles

    Image generation models create and edit images from text and reference images through APIs and open-weight models. Using them in products raises practical questions: licensing and acceptable-use terms, content moderation, likeness and copyright concerns, and disclosure. Provenance standards such as C2PA Content Credentials attach verifiable information about how an image was made.

    Why it matters: Generated media is easy to produce and hard to take back once published, so product rules and provenance matter as much as image quality.

    Example: A marketing tool generates product backgrounds, blocks prompts that name real people, keeps the provider's content credentials on exported images and labels them as AI-generated.

    Watch out: Removing provenance metadata during image processing, and ignoring a model's usage terms. Check: confirm your pipeline preserves provenance data and that your use complies with the model's licence.

    You can now: call an image generation API; apply moderation and usage rules; preserve and explain content provenance.

  12. 12. Run and adapt models you control

    Open-weight models

    Learn first: Hosted APIs vs self-hosting

    Open-weight models publish their trained weights so you can download, run, inspect and fine-tune them on your own hardware or a provider of your choice. Running them well means understanding licences, model cards, local inference tools, quantisation and serving - and being honest about how they compare with hosted frontier models on your task.

    Why it matters: Open weights give control over data, versions and cost at scale, enable offline and on-device use, and make fine-tuning practical - all valuable for regulated or high-volume products.

    Example: A team runs a small open-weight model locally for development and unit tests, and serves a quantised version for a high-volume classification step, while keeping a hosted model for complex replies.

    Watch out: Assuming "open" means free for any use - licences vary. Check: for every open model you use, record its licence, its intended use and how it scored on your evaluation set.

    You can now: find, run and evaluate open-weight models; check their licences; choose between local, self-hosted and hosted options.

    Fresher: run a model locally with Ollama. Experienced: add quantisation, serving and fine-tuning.

    Open-weight vs closed models

    Learn first: The model landscape

    Closed models are available only through their provider's API; open-weight models can be run anywhere. Closed frontier models usually lead on hard tasks and need no infrastructure; open-weight models give control over data handling, versions and customisation, and can be cheaper at high volume or smaller scale - though "open weights" rarely means open training data or code.

    Why it matters: The decision affects privacy, cost, latency, vendor dependence and how you can adapt the model, so it should be made per workload with evidence.

    Example: A hospital keeps patient-data summarisation on a self-hosted open model inside its network and uses a hosted model only for non-sensitive scheduling text.

    Watch out: Deciding by ideology or headline benchmark scores. Check: compare candidate open and closed models on your evaluation set, with full cost and data-handling requirements written down.

    You can now: explain the trade-offs between open-weight and closed models; match each to workloads; make the decision with your own data.

    Model licences & usage terms

    Learn first: Open-weight vs closed models

    Open-weight models are released under very different licences: permissive software licences such as Apache 2.0 or MIT, custom community licences with acceptable-use policies and extra conditions, and research-only or non-commercial licences. The model card and licence file state what you may do, including commercial use, redistribution of fine-tuned versions and restricted applications. Datasets have licences too.

    Why it matters: Shipping a product on a non-commercial model, or redistributing a fine-tuned model against its terms, is a legal problem no amount of engineering fixes.

    Example: Before choosing between two similar models, a team reads both licences: one is Apache 2.0, the other forbids certain uses and requires attribution in derivatives. They pick the first for a product they will redistribute to customers.

    Watch out: Assuming a model on a public hub is free for commercial use. Check: record the licence of every model and dataset in your project, and have anything unusual reviewed before launch.

    You can now: read model licences and usage policies; identify commercial, redistribution and use restrictions; document licences for a project.

    Hugging Face Hub & model cards

    Learn first: Open-weight vs closed models

    The Hugging Face Hub hosts models, datasets and demos. Each model page has a model card describing intended use, training data, evaluation results, limitations, licence and how to run it, plus the files, versions and community discussion. You can filter by task, library, language, size and licence, and pin a specific revision when downloading.

    Why it matters: The Hub is where most open models are published, and the model card is your first source for whether a model fits - and what it was never meant to do.

    Example: Looking for a multilingual embedding model, you filter by task and language, compare model cards for evaluation results and licences, and pin the chosen model to a commit hash in your code.

    Watch out: Downloading a model without reading its card, and loading repositories that run custom code you have not reviewed. Check: pin model revisions and review any model that needs remote code before running it.

    You can now: search and compare models on the Hub; read a model card critically; download and pin a model safely.

    Running models with Transformers

    Learn first: Hugging Face Hub & model cards, Environments & dependencies

    The Transformers library loads models and tokenizers from the Hub and runs them in Python. High-level pipelines give quick results; the model and tokenizer classes give control over generation settings, devices and batching; chat models expect their own chat template, which the tokenizer applies. Hardware decides what you can run - small models on a laptop CPU, larger ones on a GPU.

    Why it matters: Running a model in code, rather than through a hosted API, teaches you what inference involves and lets you evaluate open models on your own data.

    Example: You load a small instruction-tuned model, apply its chat template to a list of messages, generate answers for your 50-question evaluation set and compare the results with a hosted model's.

    Watch out: Formatting chat prompts by hand instead of using the model's chat template, which silently lowers quality. Check: confirm the model runs on the hardware you have and that outputs look sensible on a few known inputs before a full evaluation.

    You can now: run open models with Transformers; use chat templates correctly; evaluate an open model on your own tasks.

    Local inference: Ollama & llama.cpp

    Learn first: Hugging Face Hub & model cards

    Tools such as Ollama, llama.cpp and LM Studio run quantised open-weight models on ordinary laptops and desktops, on CPU or GPU. Ollama pulls models by name and serves them through a local API, including an OpenAI-compatible endpoint; llama.cpp is the underlying C and C++ inference engine many tools build on, using the GGUF model format.

    Why it matters: Local models make development cheap and private: you can prototype, write tests and process sensitive data without sending anything to a provider.

    Example: A team runs a small model locally through an OpenAI-compatible endpoint for unit tests and offline demos, so developers do not need API keys to work on the code.

    Watch out: Expecting a small local model to match a frontier model's quality. Check: run your evaluation set locally and decide which tasks the local model is good enough for.

    You can now: install and run models locally; call them through a local API; judge where local inference fits your workflow.

    • OfficialOllama Quickstart - Ollama · Free · paid extras
    • Open sourcellama.cpp - ggml-org (GitHub) · Free
    • 3 more in the topic panel.
    Quantisation & memory maths

    Learn first: Transformer & attention intuition, Running models with Transformers

    Quantisation stores weights with fewer bits - 8 or 4 instead of 16 - to cut memory and often speed up inference, with some loss of quality that depends on the method and the task. A quick estimate: weight memory in bytes is roughly the number of parameters times bits per weight divided by eight, so an 8-billion-parameter model needs about 16 GB at 16 bits and about 4 GB at 4 bits. The key-value cache for long contexts and many concurrent users adds more on top.

    Why it matters: Memory decides which models fit on which GPUs and how many users one machine can serve. Estimating it prevents buying the wrong hardware.

    Example: A plan to serve an 8-billion-parameter model at 4 bits with 8,000-token contexts for four users needs about 4 GB for weights plus several GB of cache plus runtime overhead, so a 16 GB GPU fits it with headroom.

    Watch out: Forgetting the cache and overhead, and assuming quantisation never hurts quality. Check: compare the quantised and full-precision model on your evaluation set, and measure actual memory use under load.

    You can now: estimate weight and cache memory; choose a quantisation level; measure the quality cost of quantising.

    Serving: vLLM & compatible APIs

    Optional on the Fresher track

    Learn first: Quantisation & memory maths, SDKs, raw HTTP & compatible APIs

    Serving engines such as vLLM run open models for many concurrent users with techniques like continuous batching and efficient key-value cache management, and expose an OpenAI-compatible HTTP API. Other options include SGLang, llama.cpp's built-in server and managed inference providers; check that an engine is still actively developed before you adopt it. Serving adds operational work: GPU capacity, autoscaling, monitoring, upgrades and cost per token.

    Why it matters: A model that runs in a notebook is not a service. Throughput, latency under load and cost per request decide whether self-hosting is worth it.

    Example: A team serves a quantised model with vLLM on one GPU, load-tests it at 20 concurrent users, records p95 latency and tokens per second, and compares the cost per thousand requests with a hosted API.

    Watch out: Measuring speed with one request at a time. Check: load-test at the concurrency you expect and record latency percentiles, throughput and GPU memory.

    You can now: serve a model with an OpenAI-compatible API; load-test it; compare self-hosted and hosted cost per request.

    Fine-tuning & adaptation

    Learn first: Running models with Transformers, Building an eval dataset

    Fine-tuning continues training a model on your own examples to change its behaviour: a consistent format, a narrow task done well by a smaller model, a specific style. It is powerful and expensive in data, evaluation and maintenance, so it comes after prompting and retrieval have been tried and measured.

    Why it matters: Choosing fine-tuning for the wrong reason - usually to add knowledge - wastes weeks; choosing it for the right reason can cut cost and latency substantially.

    Example: A high-volume ticket classifier with 40 labels reaches 82% with a long few-shot prompt on a large model; a small model fine-tuned on 4,000 labelled tickets reaches 94% at a fraction of the cost per call.

    Watch out: Fine-tuning without a held-out evaluation set and a baseline to beat. Check: before training, write down the baseline score, the target and how you will evaluate the tuned model.

    You can now: decide when fine-tuning is justified; prepare data and run a parameter-efficient fine-tune; evaluate the result against a baseline.

    Fresher: understand the decision; the hands-on parts can wait. Experienced: run one small LoRA fine-tune end to end.

    Prompting, RAG or fine-tuning?

    Learn first: The RAG pipeline end to end, Few-shot & step-by-step prompting

    Use prompting first: it is cheapest to change. Use retrieval when answers depend on facts that change, must be cited or are too large for a prompt. Use fine-tuning to change behaviour - a format, a style, a narrow skill - especially to let a smaller model do a task reliably at high volume. The approaches combine: a fine-tuned model can still use retrieval.

    Why it matters: The most common mistake is fine-tuning to add knowledge, which is costly, goes stale and cannot cite its sources.

    Example: "Answer using this week's price list" needs retrieval. "Always output our 12-field incident format" may need only a schema and examples. "Classify two million messages a day into 40 internal codes cheaply" is a good fine-tuning candidate.

    Watch out: Deciding without measuring the simpler options. Check: for any fine-tuning proposal, show the prompting and retrieval baselines on the same evaluation set and the expected gain.

    You can now: choose between prompting, retrieval and fine-tuning for a requirement; combine them; justify the choice with measurements.

    Supervised fine-tuning & LoRA

    Optional on the Fresher track

    Learn first: Prompting, RAG or fine-tuning?, Running models with Transformers

    Supervised fine-tuning trains a model on example inputs and ideal outputs. Parameter-efficient methods such as LoRA train small adapter weights instead of the whole model, which cuts memory and cost; QLoRA does the same on a quantised base model. Libraries such as TRL and PEFT run this on open models. Some cloud platforms also offer managed tuning for selected models, but self-serve tuning has narrowed at several providers, so check availability before you plan on it.

    Why it matters: LoRA makes fine-tuning possible on a single GPU and keeps adapters small enough to swap per task or customer.

    Example: A team fine-tunes a small open model with LoRA on 3,000 examples of its incident-report format, merges nothing, loads the adapter at serving time and compares the result with the base model on a held-out set.

    Watch out: Overfitting to a small dataset, and training on data you are not allowed to use. Check: keep a held-out evaluation set, watch validation loss during training and compare against the untuned baseline.

    You can now: explain full fine-tuning, LoRA and QLoRA; run a small supervised fine-tune; load and evaluate an adapter.

    Preparing training data

    Optional on the Fresher track

    Learn first: Supervised fine-tuning & LoRA

    Fine-tuning data is a set of examples in the model's chat or instruction format, each showing the input and the exact output you want. Quality beats quantity: consistent labels, diverse inputs, correct outputs, no personal data you may not use, duplicates removed and a held-out split kept for evaluation. Every inconsistency in the data becomes an inconsistency in the model.

    Why it matters: Most disappointing fine-tunes are data problems - contradictory examples, a skewed class mix or test cases leaking into training.

    Example: Before training a classifier, the team deduplicates 6,000 tickets, finds 300 with conflicting labels, has them relabelled, balances rare classes and holds out 600 tickets the model never sees.

    Watch out: Training on model-generated examples without checking them, and leaking evaluation data into training. Check: review a random sample of training examples by hand and confirm no evaluation example appears in the training set.

    You can now: build a fine-tuning dataset in the right format; clean, balance and deduplicate it; split it correctly for evaluation.

    Evaluating a tuned model

    Optional on the Fresher track

    Learn first: Supervised fine-tuning & LoRA, Comparing prompts & models fairly

    A tuned model must beat its baseline on a held-out set that reflects real traffic, without breaking things the base model did well. Evaluate the target task, check general abilities and safety behaviour you rely on, compare cost and latency, and test on new data over time, because real inputs drift away from the training set.

    Why it matters: Fine-tuning can raise the target metric while quietly damaging refusals, formatting or other skills - and you only see it if you test for it.

    Example: A tuned classifier gains eight points on the target task but starts answering out-of-scope questions it used to refuse; adding refusal examples to the training data and a refusal check to the evaluation fixes it.

    Watch out: Evaluating on data drawn from the training set. Check: report target-task, regression and safety results for base and tuned models side by side on held-out data.

    You can now: design an evaluation for a tuned model; detect regressions and safety changes; decide whether a tuned model ships.

    Preference tuning: DPO & RLHF

    Optional on the Fresher track · Optional on the Experienced track

    Learn first: Supervised fine-tuning & LoRA

    Preference tuning teaches a model which of two responses is better. Reinforcement learning from human feedback trains a reward model on preference data and then optimises the model against it; direct preference optimisation (DPO) and related methods learn from chosen-and-rejected pairs directly, without a separate reward model. These methods shape tone, helpfulness and safety in most assistant models.

    Why it matters: You rarely need preference tuning for applied work, but understanding it explains why assistants behave as they do and when supervised fine-tuning is not enough.

    Example: A team that has fine-tuned a support model collects pairs of good and bad replies chosen by agents and runs a small DPO step to make answers more concise without losing required details.

    Watch out: Collecting inconsistent preferences, which teach the model noise. Check: measure agreement between the people labelling preferences before training on them.

    You can now: explain RLHF and DPO; prepare preference pairs; judge when preference tuning is worth trying.

    Pretraining & distributed training

    ResearchOptional on the Fresher track · Optional on the Experienced track

    Learn first: Transformer & attention intuition, Supervised fine-tuning & LoRA

    Pretraining builds a model from scratch on huge text corpora and, at scale, needs distributed training across many GPUs with data, tensor and pipeline parallelism, careful data curation and large budgets. Most AI engineers never pretrain a production model, but building a tiny language model yourself is an excellent way to understand what the models you use are doing.

    Why it matters: This is a research and infrastructure specialism. It is optional on this roadmap, and worth exploring if you are drawn to model internals or ML engineering roles.

    Example: You train a small character-level or GPT-style model on a public text dataset on a single GPU, watch the loss fall and see generations improve, then read how the same ideas scale across thousands of GPUs.

    Watch out: Spending months here before you can ship an application. Check: do this after the core stages, and keep a clear goal - understanding, research or a career move into model training.

    You can now: explain how pretraining works; train a tiny model end to end; describe the main forms of distributed training.

  13. 13. Run it in production

    Deployment, reliability & cost

    Learn first: Build a small API service, Regression tests in CI

    Production turns a working feature into a dependable service: deployed repeatably, observable through traces and metrics, protected by timeouts, limits and fallbacks, released gradually with a way back, and run within a known cost per request. Model features add their own concerns - provider outages, rate limits, changing model behaviour and quality that drifts without errors.

    Why it matters: Most AI incidents are operational: a provider slowdown with no timeout, a cache that leaked answers, a prompt change with no rollback, a bill nobody was watching.

    Example: A summarisation service runs in containers behind a queue, traces every model call with tokens and cost, falls back to a second model on provider errors, ships prompt changes behind a flag to 5% of traffic first, and alerts when its error or refusal rate rises.

    Watch out: Monitoring only errors and latency while quality silently degrades. Check: you can answer, for yesterday, how many requests failed, what they cost and how good the answers were.

    You can now: deploy and operate an AI service; trace and monitor quality, latency and cost; roll changes out and back safely.

    Fresher: deploy one small service with logging and a budget. Experienced: concentrate on rollouts, fallbacks and quality monitoring.

    Deploying an AI service

    Learn first: Build a small API service, Config, env vars & secrets

    Package the service in a container or a serverless function, configure it through environment variables and a secrets manager, add health checks and timeouts at every layer, and run long or heavy work - batch extraction, indexing, evaluations - in background workers fed by a queue rather than inside web requests. Keep a staging environment that uses the same configuration with test keys and data.

    Why it matters: Model calls are slow and variable compared with ordinary requests, so architecture choices such as queues and timeouts decide whether the service stays responsive.

    Example: Uploading a 200-page document returns immediately with a job id; a worker parses, chunks and indexes it, updates the job status and the user is notified when it is searchable.

    Watch out: Running long model jobs inside HTTP requests that hit platform timeouts. Check: deploy to staging from a clean build, run a smoke test and confirm secrets come from configuration, not the image.

    You can now: containerise and deploy an AI service; separate request handling from background work; configure environments and secrets for deployment.

    Tracing & observability

    Learn first: Errors, logging & debugging, Usage, rate limits & budgets

    Tracing records each step of a request - model calls, retrieval, tool calls - with timing, model and prompt version, token usage, cost and redacted inputs and outputs, linked by a trace id. OpenTelemetry defines vendor-neutral conventions for generative AI telemetry, and specialised LLM observability tools add trace views, datasets and evaluations on top.

    Why it matters: When an answer is wrong, slow or expensive, the trace shows which step caused it. Without traces, debugging an AI system is guesswork.

    Example: A complaint about a wrong answer comes with a trace id; the trace shows retrieval returned an archived document and the model cited it, so the fix goes to ingestion rather than the prompt.

    Watch out: Logging full prompts with personal data and keeping them forever. Check: confirm traces carry model version, prompt version, tokens and latency, and that personal data is redacted and retention is set.

    You can now: instrument model, retrieval and tool calls; follow a request through its trace; balance observability with privacy.

    Caching responses safely

    Learn first: Prompt caching & stable prefixes, Tenant isolation & access control

    Caching avoids paying twice for the same work. Exact-match caching stores responses keyed by the full input and settings; semantic caching returns a stored answer for a similar question, which saves more but risks wrong answers when similar questions differ in what matters. Provider prompt caching is separate: it reuses processed prompt prefixes inside the provider. Every cache key must include everything that changes the answer - user or tenant, model, prompt version and data version.

    Why it matters: A cache can cut cost and latency dramatically, and a badly keyed cache can serve one customer's answer to another.

    Example: A cache for order-status answers is keyed by tenant, user, order id and prompt version, expires after five minutes because order status changes, and has a test proving user B never receives user A's cached answer.

    Watch out: Caching personalised or time-sensitive answers under a key that ignores the user or the time. Check: list every input that changes the answer and confirm each is in the key or covered by expiry.

    You can now: choose between exact, semantic and prompt caching; design safe cache keys and expiry; test caches for cross-user leakage.

    Queues, concurrency & rate limits

    Learn first: Async I/O & concurrency, Timeouts, retries & backoff

    Providers limit requests and tokens per minute, so high-volume systems schedule work: queues for background jobs, worker pools with a concurrency limit, token-aware rate limiting, priority lanes so interactive users are not stuck behind batch jobs, and backpressure that slows intake when the provider pushes back.

    Why it matters: Without scheduling, a large batch job exhausts the rate limit and every interactive user gets errors at the same time.

    Example: Nightly re-indexing runs through a queue with eight workers and a token budget per minute, while chat requests use a separate, higher-priority allowance - so the batch slows down instead of breaking chat.

    Watch out: Each process retrying independently, multiplying load during an incident. Check: load-test with batch and interactive traffic together and watch both latency and 429 rates.

    You can now: design queues and worker pools for model calls; enforce rate and token limits; protect interactive traffic from batch work.

    Latency budgets

    Learn first: Streaming responses, Tracing & observability

    Users experience latency as time to first useful output and time to completion. Break the total into a budget per step - retrieval, reranking, model time to first token, generation, tools, post-processing - and reduce the largest: stream output, shorten prompts and outputs, use a smaller model for simple steps, run independent calls in parallel, cache, and keep services close to the provider's region.

    Why it matters: Latency decides whether people use a feature. A one-second improvement at the 95th percentile matters more than a faster average.

    Example: A RAG assistant's p95 is 7.8 seconds. Traces show 3 seconds in a sequential reranker and 4 seconds generating long answers; parallelising and capping the answer length brings p95 to 3.9 seconds with no loss on the evaluation set.

    Watch out: Reporting only averages and optimising steps that are not the bottleneck. Check: measure p50, p95 and p99 per step from traces before and after each change.

    You can now: set a latency budget per step; measure percentiles from traces; apply the right latency fix for each bottleneck.

    Cost: routing, batching & budgets

    Learn first: Usage, rate limits & budgets, Quality, latency & cost trade-offs

    Cost control starts with measuring cost per request, per user and per feature, then applying the levers: route easy requests to small models and escalate only hard ones; trim context and cap output length; use prompt caching; send non-urgent work to batch APIs, which several providers price lower in exchange for slower turnaround; and enforce budgets with alerts and hard stops.

    Why it matters: A feature's unit cost must fit its value. Optimising blindly can also damage quality, so every cost change needs an evaluation.

    Example: Routing 80% of tickets to a small model and 20% to a large one cuts cost by more than half - but only after the evaluation shows the routed system still passes every blocking metric and fallbacks to the large model stay rare.

    Watch out: Cost savings that quietly drop a required step or condition. Check: report cost per 1,000 requests together with the evaluation scores for every optimisation.

    You can now: measure unit cost; apply routing, caching, batching and trimming; prove a cost change kept quality.

    • OfficialBatch API - OpenAI · Free · paid extras
    • OfficialPrompt caching - Anthropic · Free · paid extras
    • 4 more in the topic panel.
    Fallbacks & graceful degradation

    Learn first: Timeouts, retries & backoff, Structured outputs & JSON Schema

    Decide in advance what happens when a model is unavailable, slow, refuses or returns invalid output: retry within limits, switch to a fallback model that has passed the same output contract tests, use a simpler non-AI path, queue the work for later or hand over to a person - and tell the user honestly. Circuit breakers stop sending traffic to a failing provider for a while.

    Why it matters: Providers have outages and slowdowns. A designed fallback keeps the product useful; an improvised one can return broken output or duplicate actions.

    Example: When the primary provider errors, the triage service switches to a second model that is tested against the same schema; if both fail, tickets go to the human queue with a banner explaining the delay.

    Watch out: A fallback model that has never been evaluated against your contract, and treating fallback text as a successful result. Check: run the evaluation suite against every fallback path and test the circuit breaker in staging.

    You can now: design fallback chains; evaluate fallbacks against the same contract; degrade gracefully and communicate honestly.

    Canaries, shadow tests & rollback

    Learn first: Regression tests in CI, Deploying an AI service

    Release model, prompt and retrieval changes gradually. A shadow test runs the new version on live traffic without showing users its output, so you can compare; a canary sends a small share of real traffic to the new version with automatic checks; feature flags switch versions instantly. A rollback plan restores the whole compatible set - prompt, parser, schema, index and model - not just one part.

    Why it matters: Offline evaluations miss some real-world failures. Gradual rollout limits the damage and gives you evidence before a full release.

    Example: A new prompt runs in shadow for two days, then goes to 5% of traffic with gates on schema failures and escalation rate; a spike in one language triggers automatic rollback to the previous prompt and parser together.

    Watch out: A canary whose sample differs from the full traffic, and rolling back the prompt but not the parser that expects it. Check: compare the canary's traffic mix with production and rehearse the rollback.

    You can now: run shadow tests and canaries; define rollout gates; roll back a coordinated set of changes safely.

    Monitoring quality in production

    Learn first: Tracing & observability, Model-graded evals & their limits

    Quality can fall with no errors in the logs, so monitor signals of it: user feedback, escalations to humans, refusal and fallback rates, schema-validation failures, retrieval misses, and scheduled evaluation of a sample of real traffic with code checks and calibrated judges. Watch inputs too - new topics, languages or document types are early signs of drift.

    Why it matters: Users rarely report bad answers; they stop using the feature. Monitoring catches the decline before they leave.

    Example: A weekly sample of 200 conversations is scored for groundedness; a drop from 95% to 88% is traced to a new product line missing from the index, which is added before complaints arrive.

    Watch out: Dashboards that show only averages across all users. Check: break quality signals down by segment and alert on sustained changes, not single spikes.

    You can now: choose quality signals for a live feature; run evaluation on sampled traffic; detect and investigate drift.

    AI gateways & model routers

    Optional on the Fresher track · Optional on the Experienced track

    Learn first: Fallbacks & graceful degradation, Cost: routing, batching & budgets

    An AI gateway sits between your services and model providers to centralise keys, routing, retries and fallbacks, caching, rate limits, budgets and logging. Options include open-source proxies, cloud platform gateways and hosted routers that reach many providers through one API. They simplify multi-provider setups but add a dependency and another place your data passes through.

    Why it matters: As usage grows across teams, a gateway prevents every service from reimplementing keys, limits and fallbacks inconsistently.

    Example: A company routes all model traffic through one gateway that enforces per-team budgets, strips personal data from logs and switches providers during an outage without code changes.

    Watch out: Sending regulated data through a third-party router without checking its data handling. Check: review the gateway's logging, retention and data-processing terms, and test its fallback behaviour.

    You can now: explain what gateways and routers do; decide when one is worth adding; configure routing, budgets and fallbacks centrally.

  14. 14. Prove it with working software

    Projects & career evidence

    Hiring

    Learn first: Evaluation & testing

    Employers trust evidence they can check: working projects with tests and evaluations, write-ups that show how you made decisions, contributions to real codebases, and scored assessments. The aim of this stage is to turn what you learned into a small number of artefacts that show judgement, not a long list of tools.

    Why it matters: Many candidates show the same tutorial chatbot. A project that measures its own failures and explains its trade-offs stands out immediately.

    Example: A portfolio with three projects - a cited policy assistant with a retrieval evaluation, a structured extraction service with a cost report, and a bounded tool-using workflow with an approval step - each with a two-page write-up and a short demo video.

    Watch out: Collecting certificates instead of building and measuring. Check: for each project, can a reviewer run it, see its evaluation results and read what failed?

    You can now: plan a portfolio that shows judgement; write up projects convincingly; back your skills with verifiable evidence.

    Fresher: build the capstone and one more project. Experienced: lead with projects from your domain and production experience.

    Capstone: one feature, end to end

    Learn first: The RAG pipeline end to end, Building an eval dataset, Deploying an AI service

    Build one complete feature that touches the core of this roadmap: a clear problem and user, ingestion or tools as needed, structured outputs, an evaluation set with results, security controls for untrusted input, a cost and latency estimate, a deployment and a write-up of what you would do next. Small and finished beats big and half-built.

    Why it matters: An end-to-end build shows you can make all the parts work together, which is exactly what the job asks for.

    Example: A policy question-answering service for a fictional company: parsed and chunked policies, permission-aware retrieval, cited answers or abstention, 80 evaluation questions, an injection test set, traces, a budget per request and a deployed demo.

    Watch out: Spending all the time on the interface and none on evaluation. Check: your README shows evaluation results, known failure modes, cost per request and how to run the tests.

    You can now: scope and build a complete AI feature; evaluate, secure and deploy it; present it as evidence.

    Portfolio projects that show judgement

    Learn first: Capstone: one feature, end to end

    Strong portfolio projects solve a believable problem with realistic, messy data, measure quality with an evaluation set, handle failures deliberately, state their costs and limits, and explain the decisions behind them. Two or three such projects, each with a repository, a short demo and a write-up, are worth more than many tutorial clones.

    Why it matters: Reviewers skim. A project whose first screen shows the problem, the evaluation result and one interesting failure gets read.

    Example: Instead of "chat with PDF", a project compares three chunking strategies on 60 labelled questions, shows the winner improved hit rate from 68% to 87%, and explains the remaining failures.

    Watch out: Projects that cannot run without your private keys, and claims without numbers. Check: ask someone to clone and run your project from the README alone, and fix whatever stops them.

    You can now: choose projects that demonstrate judgement; present results with evidence; make projects reproducible for reviewers.

    Design docs & eval reports

    Learn first: Comparing prompts & models fairly

    Engineers communicate decisions in writing. A design document states the problem, constraints, options considered, the decision and its trade-offs; an evaluation report states the dataset, metrics, results by segment, failure analysis and limits; a decision memo gives a recommendation, the evidence and what would change it. Short and specific beats long and general.

    Why it matters: AI decisions involve uncertainty. Writing that separates what was measured from what is assumed builds trust with reviewers, managers and interviewers.

    Example: A one-page memo recommends keeping the current model: the cheaper candidate saves 40% but misses the grounded-answer gate in Hindi by six points; it names the next experiment and the owner.

    Watch out: Overstating results - "99% accurate" from twenty examples. Check: every number in a write-up names its dataset and size, and every claim is marked as measured or assumed.

    You can now: write a design document and an evaluation report; state uncertainty honestly; make a recommendation a team can act on.

    Contributing to open source

    Learn first: Git & pull requests, Testing with pytest

    Open-source AI projects - SDKs, evaluation tools, retrieval libraries, MCP servers - welcome fixes, documentation, tests, examples and reproducible bug reports. Start by reading the contributing guide, picking issues labelled for newcomers, reproducing the problem, and opening a small, focused pull request with tests.

    Why it matters: A merged contribution is public, reviewed evidence that you can read unfamiliar code, follow a team's standards and communicate clearly.

    Example: You hit a bug in an evaluation library's metric, write a minimal reproduction, find the off-by-one error, add a failing test and open a pull request that the maintainers merge after one round of review.

    Watch out: Large unsolicited pull requests and low-effort changes made only for a contribution count. Check: read recent merged pull requests in the project to learn what maintainers expect before you start.

    You can now: find suitable issues in AI projects; make a focused, tested contribution; work through code review with maintainers.

    Hackathons, used well

    Optional on the Experienced track

    Learn first: Git & pull requests

    Hackathons are short, intense builds with a team, often with sponsor APIs and prizes. They are good for practising scoping under time pressure, teamwork and demos - and they become strong evidence when you carry a project forward afterwards with tests and an evaluation. Check the organiser's own page for dates, eligibility, costs and intellectual-property rules before you register.

    Why it matters: A hackathon demo alone shows energy; a hackathon project you later evaluated, secured and deployed shows engineering.

    Example: After a weekend hackathon, a team spends two more weeks adding an evaluation set, rate limiting and a write-up to their voice assistant, and that version goes into their portfolios.

    Watch out: Events listed only on aggregator sites with unclear organisers, and rules that assign your code to a sponsor. Check: confirm the organiser, dates and terms on the organiser's own site; treat any event list as time-sensitive.

    You can now: choose hackathons worth your time; scope a build for a short event; turn a hackathon project into portfolio evidence.

    Interviews: LLM system design

    Learn first: Capstone: one feature, end to end

    An LLM system design answer follows a clear order: clarify the users and the task; set requirements for quality, latency, cost, privacy and safety; sketch the architecture - retrieval, tools, models, storage; explain how you will evaluate it before and after launch; walk through failure modes and controls; then cover scale, cost and rollout. Interviewers listen for trade-offs and measurement, not framework names.

    Why it matters: Design rounds are where applied AI candidates are separated, and they reward a repeatable structure plus real examples from your projects.

    Example: For "design an assistant for HR policy questions", a strong answer covers document permissions by employee group, cited answers with abstention, an evaluation set built from real questions, escalation for sensitive cases and a budget per conversation.

    Watch out: Jumping straight to an architecture diagram without requirements. Check: practise three prompts aloud and confirm each answer covers evaluation, failure modes and cost.

    You can now: structure an LLM system design answer; discuss trade-offs with numbers; draw on your own projects as examples.

    Prove it: certification & simulation

    Learn first: Evaluation & testing, AI security & privacy

    Reading a topic and marking it done is learning progress, not proof. OneRoadmap offers three kinds of evidence: the free AI Engineer certification, which is scored and verifiable; the AI Engineer job simulation, where you write a production triage prompt and then decide from evaluation data whether a new pipeline should ship; and daily scenario challenges, which practise the decisions this roadmap teaches and add evidence over time.

    Why it matters: Recruiters can verify scored evidence in one click; they cannot verify a list of courses.

    Example: After the evaluation and security stages, a learner takes the job simulation: they write a triage prompt that returns strict JSON for every ticket, then use the supplied evaluation results to make and defend a ship-or-hold recommendation.

    Watch out: Treating completion of resources or ticked nodes as verified skill. Check: each claim on your profile links to a scored assessment, a simulation result or a project with evaluation evidence.

    You can now: choose the right OneRoadmap proof for your level; prepare for the certification and simulation; build a verifiable record of your skills.

    Switching in: use your domain

    Not on the Fresher track

    Learn first: Fresher path vs switcher path

    If you are moving into AI engineering from another field - finance, healthcare, law, support, operations, teaching - your domain knowledge is an advantage. You know which questions matter, what a good answer looks like, where errors are costly and which data is messy, which is exactly what problem framing and evaluation require.

    Why it matters: Domain-specific projects with realistic evaluation sets are rarer and more convincing than generic demos, and they point you towards employers in your field.

    Example: A former insurance claims officer builds a claims-document extractor with an evaluation set of tricky real-world cases they know from experience, and writes up the failure modes an engineer without that background would miss.

    Watch out: Abandoning your domain to compete on generic projects. Check: your first two projects should use problems and data from the field you know.

    You can now: turn domain expertise into AI projects; build domain-specific evaluation sets; position yourself for AI roles in your industry.

Prove it as you go

Reading is half of it. This roadmap is backed by a free, scored AI Engineer certification, a job simulation on a real triage system and daily scenario challenges - evidence a recruiter can check, not a claim.

Projects that show judgement

A support answerer that refuses to guess

Retrieval over a small policy set with citations, a 'no answer' path, permission filters and an evaluation set that proves unsupported answers went down.

A structured extraction service

Messy documents in, schema-valid JSON out: validation, a bounded repair loop, cost per document and a report of what still fails.

A bounded tool-using workflow

Two or three tools behind an MCP server, step and cost limits, idempotent writes, human approval for the risky step and a trace you can replay.

What most learners get wrong

Learning three agent frameworks before writing one plain API call.

Judging a prompt on the five examples it was written against.

Shipping without an evaluation set, so every change is a guess.

Trusting retrieved documents, tool results and model output as if they were instructions.

Ignoring cost and latency until the first bill or the first timeout.

Letting an agent loop run without step, time or spending limits.

Fine-tuning to add knowledge that retrieval would keep current.

A portfolio of demos with no tests, no failure handling and no write-up.

Credentials: what is free and what is paid

OneRoadmap AI Engineer certification - free

Scored deterministically and added to your Talent Graph, so a recruiter can verify it in one click.

Free courses, paid certificates

Most courses on this map are free to read or watch. Some platforms charge for a certificate of completion, and running examples can need a paid API key - each resource says which.

Cloud vendor certifications

Paid exams from cloud providers can help where a job names that cloud. They are optional; working projects and evaluation evidence matter more.

Keep going

Go deeper on the language and the systems this roadmap leans on, or specialise in agents once the core is in place.

Frequently asked questions

What does an AI engineer actually do?

An AI engineer builds product features on top of existing models: calling model APIs, designing prompts and context, retrieving the right data, letting a model use tools, and - the part that separates engineers from demos - measuring quality, controlling cost and latency, and keeping the system safe when inputs are hostile. Most of the work is ordinary software engineering with one unreliable component in the middle, and the job is making that component dependable enough to ship.

Do I need a machine learning or maths background?

Not to start. Applied AI engineering leans on software skills - Python, APIs, testing, data handling - plus a working intuition for how models behave: tokens, context, sampling, embeddings and why models hallucinate. Linear algebra and training maths matter if you move into fine-tuning or model research, which this roadmap keeps as optional branches. Learn the engineering first; go deeper into the maths when a real problem needs it.

Python or JavaScript?

Python is the safer first choice: most model SDKs, evaluation tools, data libraries and examples start there. TypeScript is a strong second language and is common for AI features inside web products. The concepts on this roadmap - structured outputs, retrieval, evaluation, tool calling - are the same in both, so pick one, get fluent, and read the other.

Do I have to pay for model APIs to learn this?

Not much, and sometimes nothing. Many providers offer free tiers or small trial credits, open-weight models run locally with tools like Ollama on an ordinary laptop for small sizes, and most exercises here work with tiny inputs. Every resource on this roadmap says whether it is free to read and whether running its examples needs a paid API key or GPU. Set a hard spending limit in any provider console before you start, and keep evaluation sets small until they need to grow.

How long does it take to become job-ready?

It depends on where you start, and nobody can promise a timeline. A developer who already ships backend code can usually work through the core of this roadmap in a few months of steady practice; someone new to programming should expect the foundations alone to take months. What matters more than hours is evidence: two or three working projects with an evaluation set, a cost estimate and a write-up of what failed and how you fixed it.

Is prompt engineering enough?

No. Prompting is one tool, and on its own it rarely survives real users. Employers look for people who can turn a prompt into a component: validated structured output, retrieval with permissions, tests that catch regressions, sensible retries, logs, a cost budget and a safe failure mode. That is why prompting is one stage here and evaluation, security and production are stages of their own.

Should I learn LangChain, LlamaIndex or an agent framework first?

Learn the concepts with plain SDK calls first, then pick one framework if a project needs it. Frameworks change quickly and hide the behaviour you need to debug - what was retrieved, what the model saw, why a tool was called. Once you can build a small retrieval pipeline and a bounded tool-calling loop by hand, any framework becomes a convenience instead of a mystery. Framework nodes on this map are outlined for that reason.

When should I fine-tune a model instead of prompting or using retrieval?

Usually later than people think. Prompting and retrieval are cheaper to change and easier to audit; retrieval is the right tool when answers depend on facts that change. Fine-tuning earns its cost when you need a consistent format or behaviour at high volume, a smaller and cheaper model to match a larger one on a narrow task, or a style examples cannot reliably teach - and only once you have an evaluation set that proves the tuned model is better.

How do I prove AI engineering skills without job experience?

Ship small, real systems and show the evidence: a repository with tests, an evaluation set and its results, a short design document with the trade-offs you made, and a note on cost and failure handling. Practise on scenario challenges that mirror the job, take a scored certification and a job simulation so a recruiter can verify your skills, and contribute fixes or evaluations to open-source AI projects. Evidence of judgement beats a list of tools.

Will AI replace AI engineers?

Coding assistants already write much of the boilerplate, and they make AI engineers faster rather than unnecessary. The parts that remain human - deciding what the system should and should not do, defining what good looks like and measuring it, spotting a retrieval leak or an injected instruction, and taking responsibility for the release - are the parts this roadmap spends the most time on.