AI Engineer

An AI engineer builds products on top of foundation models - integrating, grounding and evaluating them - rather than training models from scratch. It's a software engineering role with a new primitive, reachable from frontend, backend or full-stack.

ML engineers train and deploy models from data; AI engineers compose pre-trained models into products. Different toolchains, different failure modes - and different interviews.

Prompt and context design, API integration, retrieval pipelines, evals, cost/latency budgets and safety review - the day-to-day surface of the job, and how it changes product development itself.

LLM, inference vs training, tokens, context window, embeddings, vector database, RAG, agent, AGI vs today's AI - the working glossary you'll use in every design discussion.

Next-token prediction, sampling and temperature - and why hallucination is a property of the mechanism, not a bug to be patched.

Training frontier models costs millions; using them costs cents. The benefits (capability now, no ML team) and the trade-offs (cost per call, data governance, vendor dependency).

Anthropic's Claude, OpenAI's GPT models, Google's Gemini, Mistral, Cohere and Llama-family open models - plus cloud platforms (Azure AI, AWS Bedrock/SageMaker) that host them.

Context length, knowledge cut-off dates, modality support and speed tiers - the spec sheet that decides model fit before any benchmark does.

Hallucination, prompt sensitivity, knowledge staleness and non-determinism - designing products that stay useful when the model is wrong.

System, user and assistant roles, multi-turn state and stop conditions - the request anatomy shared by every provider's API.

Provider workbenches for iterating on prompts before writing code - parameters, comparisons and shareable experiments.

How text becomes tokens, counting them programmatically, and max-token limits - the unit your budget and context window are denominated in.

Input vs output pricing, prompt caching, batching and model routing (small model first) - AI features have a unit cost you must design, monitor and defend.

Server-sent tokens and progressive UI - the difference between an app that feels instant and one that feels broken.

Adjusting weights on your examples: strong for style and narrow tasks, usually beaten by better prompts + RAG for knowledge. Know the decision framework.

Role, task, constraints, examples, output format - a repeatable template that makes results predictable across inputs.

Examples that teach the pattern and reasoning room that improves accuracy - and when each actually helps.

Schema-constrained JSON plus validate-and-retry loops - downstream code should never parse prose.

Testing prompts against fixed input sets, diffing outputs and keeping prompts in git next to code.

Untrusted content smuggling instructions into trusted context - the signature attack of LLM apps, and layered defences for anything that reads user or web content.

Models inherit their training data's biases - testing across demographics and use cases before your users do it for you.

PII in prompts, data retention policies, end-user IDs for abuse attribution and regional processing - the questions legal will ask.

Input/output filtering, moderation endpoints, constrained outputs and human-in-the-loop for consequential actions.

Red-teaming your own app with jailbreak and injection suites before launch - finding the failure modes on your schedule, not Twitter's.

Open-weights models trade some capability for control, privacy and unit economics - the decision matrix for regulated data and offline use.

Finding models by task, reading model cards and licenses, and the Hub as the app store of open ML.

Inference endpoints, the transformers library and Transformers.js in the browser - from Hub page to running output.

Pulling and running quantised models locally with an OpenAI-compatible API - prototyping and private workloads on your own hardware.

Text mapped to vectors where distance ≈ meaning - the primitive behind semantic search, recommendations, anomaly detection and classification.

Semantic search over docs, 'similar items' recommendations, outlier detection and cheap zero-shot classification - one primitive, many products.

Hosted APIs vs open Sentence Transformers - dimension, speed and cost trade-offs, and why you must never mix models in one index.

pgvector, Chroma, Pinecone, Qdrant, Weaviate, FAISS and friends - pick one, learn indexing and filtering deeply; the concepts transfer.

Cosine similarity, top-k retrieval, metadata filters and hybrid (keyword + vector) search - precision work that decides RAG quality.

Retrieval for knowledge (fresh, auditable, cheap to update); fine-tuning for behaviour - the trade-off every 'chat with our data' project must get right.

Chunk → embed → store → retrieve → generate with citations. Each step has failure modes (bad chunks, stale index, irrelevant top-k) and fixes.

Size, overlap and structure-aware splitting so retrieval returns answers, not noise.

Orchestration frameworks vs calling SDKs directly - what they abstract, what they hide, and when plain code is clearer.

Declaring functions the model can call, executing them and returning results - the primitive underneath every agent.

Reason → act → observe cycles, stop conditions and budgets - and why an unbounded loop is an outage waiting to happen.

Deterministic pipelines beat autonomous agents for most tasks - choose the simplest structure that works, escalate autonomy only when needed.

Conversation windows, summarisation and external state - keeping long tasks coherent past the context limit.

The Model Context Protocol standardises connecting models to tools and data - the plumbing layer under modern agent apps.

Vision-capable models reading screenshots, documents and photos - OCR-free document extraction is a product category now.

Text-to-image APIs and prompt technique - plus the licensing and safety questions that ship with generated media.

Transcription (Whisper-class) and voice synthesis - the interface layer for voice-first products.

Combining vision, audio and text in one flow (e.g. meeting recording → transcript → summary → action items) with your framework of choice.

Golden sets, LLM-as-judge with spot-check calibration and regression evals in CI - you can't improve what you don't measure.

Tracing prompts, tokens, latency and tool calls in production - failures you can debug instead of shrug at.

Rate limits, timeouts and idempotent retries with backoff - network engineering applied to model calls.

AI code editors and agentic CLIs (Claude Code, Cursor, Copilot) - using AI to build AI, and knowing when to trust it.

Ship one AI feature end to end - RAG app, agent or copilot - deployed, evaluated and documented.

Take the OneRoadmap AI Engineer certification to verify the skills recruiters are now screening for.