About the role
structured by ORIAI Engineer – AI Agents (Healthcare)BengaluruAI Team /On-siteapply for this jobWe’re looking for an AI Engineer to help us design, evaluate, and scale our next generation of LLM-powered agents. This role is deeply technical, but also product and customer-oriented: you’ll build datasets, run evaluations, improve model…
What you will do
- Build, curate, and maintain agentic workflows along with datasets for training and evaluation.
- Run systematic LLM evaluations, track regressions, and ensure models meet quality bars.
- Define and implement LLM performance metrics(e.g., correctness, latency, hallucination control, safety).
- Work closely with Product and Customer-facing teams, experiment with prompting techniques, fine-tuning datasets, retrieval strategies, and model configurations and ensure agent behaviour aligns with user expectations.
- Develop internal tooling to speed up evaluation, annotation, and iteration cycles.
What they are looking for
- 2–5 years experience in software engineering, with strong fundamentals in building reliable, scalable systems.
- 1–2 years of experience working on AI-backed products(LLM engineering, evaluations, prompting, or agent development).
- Hands-on experience with LLMs, prompt engineering, evaluation frameworks, or dataset creation.
- Strong understanding of how LLMs work: prompting, fine-tuning, evaluation, guardrails, and system design.
- Solid software engineering skills and comfort with APIs and cloud environments.
- Experience building internal tools or automation pipelines.
- Ability to think from the customer backward: empathetic, detail-oriented, and outcome-driven.
- Excellent communicator who collaborates well across cross-functional teams.
- Fast learner who loves turning cutting-edge research into practical, dependable systems.
Nice to have
- Bonus: experience with voice AI, speech technologies, or real-time agent systems.
Benefits
- Small team at a fast-growing engineering-first startup
- Work with engineers across the globe with experience in video at places like Facebook and Hotstar
- Freedom to set your own goals and grow as an individual contributor or as a team leader
Full posting text
We’re looking for an AI Engineer to help us design, evaluate, and scale our next generation of LLM-powered agents. This role is deeply technical, but also product and customer-oriented: you’ll build datasets, run evaluations, improve model performance, and ensure our agents deliver real value in real workflows.
As an AI Engineer, you’ll work across Engineering, Product, and Customer teams to continuously improve our agent quality, reliability, and speed. You won’t just build agents — you’ll build the systems, metrics, and feedback loops that make those agents better over time.