Back to ML Engineer Job Simulation

Prerequisites & resources - ML Engineer simulation

There is nothing to install and nothing to upload. Both tickets are answered in the browser: five questions and a written note each. No notebook is submitted and no code of yours is executed.

Everything you need to answer is printed in the tickets - a dataset card, a training notebook excerpt, a confusion matrix and a set of production constraints. Bring a calculator for one question.

What you should already know

You do not need production ML experience. You need to be able to read someone else's setup and say what is wrong with it.

  • Supervised classification basics - train/validation splits, what a model fits
  • Classification metrics - precision, recall, and how to read a confusion matrix
  • Class imbalance - why accuracy stops being informative when one class is rare
  • Data leakage - both kinds: a feature that encodes the label, and preprocessing fitted before the split
  • scikit-learn shapes - fit_transform, train_test_split, cross-validation, roughly what a StandardScaler does
  • A rough sense of deployment - that a model has to be served, and that serving can differ from training

You do not need deep learning, PyTorch, or MLOps tooling experience.

Reference material

Reading these is allowed and encouraged - looking things up is part of the job.

  • Confusion matrix, precision and recall - scikit-learn User Guide → Classification metrics
  • Why accuracy misleads on imbalanced data - Google ML Crash Course → Classification: Accuracy, Precision, Recall
  • Data leakage - scikit-learn User Guide → Common pitfalls: Data leakage, especially the section on fitting transformers before splitting
  • Pipelines - scikit-learn → Pipeline, and why it exists (it is the fix for one of ML-101's defects)
  • Time-based validation - scikit-learn → TimeSeriesSplit
  • Training/serving skew - Google's Rules of Machine Learning, Rule #29
  • Shadow deployment - any reputable write-up of shadow mode vs canary releases

Ground rules

  • Answer from the material given. Generic ML advice scores badly. Every question points at something concrete in the ticket - a specific feature, a specific line, a specific constraint.
  • One question requires arithmetic. Read the confusion matrix carefully; it is easy to compute the wrong one of precision and recall.
  • Using AI assistance is allowed, exactly as on the job. It will not carry you here: the numbers come from this brief, and the notes are graded on reasoning you would have to defend in a review.
  • Two free submissions per ticket. Read the feedback before resubmitting.
  • Write the notes for the stated reader. ML-101's note goes to a non-specialist product lead; ML-102's plan will be challenged. The rubrics reward writing that survives both.