Experience Not stated in the posting - worth applying if the stack matches
About the role
structured by ORIML Research Engineer; Pre-training (LLMs)Bengaluru · Full-time · Experience: 2–4 yrsAbout EkaCare and the missionEkaCare is India's connected healthcare platform: an EMR that doctors run their practices on, a personal health record used by millions of Indians, and one of the deepest integrations with India's ABDM…
What you will do
- Design and run CPT/mid-training ablation ladders at proxy scale, the experimental backbone that decides the real run
- Own data-mix and curriculum empirics: medical vs general, Indic vs English, replay ratios (~70% design point), annealing schedules
- Debug training at scale: loss spikes, precision issues, dataloader stalls, checkpoint pathologies
- Build per-stage eval hooks so every CPT phase has a scoreboard, not a vibe
- Run tokeniser, long-context and MoE-health experiments (routing balance, expert utilisation)
What they are looking for
- 2–4 years in ML with pretraining or CPT you personally ran at ≥1B scale (ideally ≥7B, 100B+ tokens) — the recipe was yours to break and fix
- Fluency with Megatron/NeMo/TorchTitan-class trainers and distributed fundamentals (TP/PP/DP, mixed precision)
- Empirical rigour: you design ablations that answer questions
- You read papers fast and implement faster
Nice to have
- MoE training exposure; scaling-laws mindset
- Indic-language or domain-specific (medical/legal/code) pretraining
- Kernels curiosity — you've opened a profiler and enjoyed it
Benefits
- Open source, with your name on it: weights and technical reports ship publicly
- India-scale mission: models for a billion people in their own languages
- Compute that’s rare to fine: dedicated multi-node H200 training under a national grant
- Small senior team: you work with the people who own the recipe
- A live deployment path: Government institutes, EkaCare's doctors and patients use what you ship
pre-trainingLLMsCPTdistributed fundamentalsTPPPDPmixed precisionMegatronNeMoTorchTitanH200
Full posting text
Experience: 2 - 4