Experience Senior · 6+ yrs (5+ years)
About the role
structured by ORIKernel Optimisation & Inference EngineerBengaluru · Full-time · 2–5 yrsSomewhere between the model and the silicon, 10–20% of a training budget goes missing. Your job is to go get it back, and then make the same model fast enough to run in a clinic, and small enough to run on a phone.About EkaCare and the…
What you will do
- Profile training and inference workloads and hunt utilisation gaps across kernels, memory and comms.
- Write and tune CUDA kernels where existing ops leave real performance on the table, and know when they don't.
- Optimise MoE-specific paths: grouped GEMMs, all-to-all communication, expert load imbalance.
- Build the fast inference path: vLLM-class serving, continuous batching, prompt/prefix caching for clinical-context workloads, speculative decoding.
- Own quantisation for the 2B/4B variants (AWQ/GPTQ-class, fp8) — with eval-parity verification, not just perplexity.
What they are looking for
- 2–5 years in GPU performance work; you've profiled real workloads and shipped optimisations with before/after numbers you can defend.
- Working fluency in CUDA, and memory-hierarchy reasoning (coalescing, occupancy, SRAM tiling; you can explain *why* FlashAttention is fast).
- Hands-on with a modern serving stack (vLLM, TensorRT-LLM, SGLang or similar) beyond just running it.
- Measurement discipline: you profile before optimising and verify correctness after.
Nice to have
- fp8 experience on H100/H200-class hardware; torch.compile/inductor internals.
- Quantisation research or on-device/mobile inference experience.
- Open-source kernels or serving contributions.
Benefits
- Medical Insurance & Accidental Insurance
- Maternity & Paternity Benefits
- PF, Gratuity, & Leave Encashment
- Salary Advance Policy
CUDAGPU PerformanceKernel OptimisationQuantisationMemory Hierarchy ReasoningSpeculative DecodingContinuous BatchingvLLMTensorRT-LLMSGLangtorch.compileAWQGPTQ
Full posting text
Experience: 2 - 5