AI/ML Engineer — Data, Evaluation & Model Improvement

SecNinjaz Technologies LLP

Openings: 1 | Experience: around 2 years

Location / work mode: Delhi

About the role

Build the data, evaluation and model-improvement capability behind AI-powered cybersecurity systems.

Turn reviewed security work into trustworthy datasets, measure performance on unseen cases, and run

reproducible language-model post-training experiments.

You will develop and test Python pipelines, working with cybersecurity specialists and an agent/runtime

engineer. You need strong applied AI/ML engineering and practical cybersecurity ability to judge

evidence, understand labels and determine whether an experiment improves a security workflow.

What you will do

• Turn approved workflow traces, tool outputs, failures, expert corrections and verified outcomes into

versioned datasets, with provenance, permitted-use tracking, sensitive-data handling, deduplication and

separate training, validation and held-out test cases, grouping related applications, engagements and

near-duplicate traces to prevent leakage between splits.

• Work with security specialists to define tasks, labels, negative cases and independently checked

outcomes; investigate ambiguous labels, false positives and data leakage.

• Build repeatable evaluations for task success, finding quality, tool use, regressions, latency and cost.

Compare model and harness changes against versioned baselines, isolate changes where practical,

and document interactions, uncertainty and evaluation limitations.

• Run reproducible language-model post-training experiments using supervised fine-tuning,

preference-based methods or reinforcement learning where appropriate; explain the chosen method

and investigate misleading reward signals.

• Jointly define versioned trace and tool interfaces, evaluation criteria and release checks with the

agent/runtime engineer and cybersecurity specialists. Security specialists review domain labels and

validate findings.

• Work with the runtime engineer on controlled execution environments and feedback for evaluation and

training experiments. Keep operational memory, training data and held-out evaluation cases distinct.

• Package datasets, configurations, model checkpoints and results so another engineer can reproduce

the work; document failures and recommend whether an improvement warrants adoption.

• Integrate accepted models into the agent workflow with compatibility tests, regression checks and

versioned rollout/rollback, including private or self-hosted deployment where required.

SecNinjaz Technologies LLP 1 What you should bring

• Around two years of hands-on development experience; strong Python, data processing, Git, debugging

and automated testing.

• Practical AI/ML knowledge: training versus inference, loss functions, overfitting, generalisation, data

quality, metrics and experimental comparison; understanding of LLM behaviour, prompting, retrieval

and tool use.

• A reproducible language-model fine-tuning or post-training project covering data preparation, a

baseline, held-out results and failure analysis. A well-executed prototype or implemented coursework

qualifies; be ready to explain and modify your implementation.

• Working knowledge of dataset provenance, label review, deduplication and leakage prevention.
• Reinforcement-learning fundamentals: environments, actions, rewards, evaluation and the risk of

rewarding the wrong behaviour.

• Demonstrated practical cybersecurity ability: reason about authentication, authorization, trust

boundaries and common web/API or code weaknesses; interpret tool output, reproduce and validate a

finding, reject a false positive and verify a fix in a controlled environment.

• Ability to integrate an LLM into a Python application or agent workflow, work with APIs and Linux tooling,

and explain experimental results clearly.

Relevant professional work, labs, research and personal projects can demonstrate these skills.

Good to have

• Preference optimisation, RL fine-tuning, distillation or comparison of post-training methods.
• Experiment tracking, dataset versioning and training or inference performance work.
• Open-weight model serving, quantisation or private deployment.
• Deeper security research, detection engineering or security evaluation experience.

Initial outcomes, with the team

• Establish a permitted pipeline from workflow traces to reviewed, versioned datasets and an evaluation

baseline.

• Run a bounded post-training experiment and compare it with the base model on held-out security tasks,

documenting limitations and failures.

• Demonstrate integration of the candidate model in the agent workflow and recommend adoption or

rejection using quality, reliability and cost evidence.

Selection process

A discussion of an existing post-training project and a short practical exercise using synthetic or sanitized

security data. You will interpret evidence, review labels and permitted use, identify leakage risks,

implement a small data or evaluation improvement, and explain a baseline comparison and adoption

decision.

The exercise does not require a large new training run. AI tools may be used; explain your contribution

and be ready to debug your solution. Nothing is run against real systems.

Apply

Send your CV and a relevant fine-tuning/post-training project link, or a short technical write-up covering

your contribution, data preparation and evaluation, to [application contact/link] . Use "AI/ML Engineer

— Data, Evaluation & Model Improvement" as the application subject.

All security work at SecNinjaz is authorized and operates under agreed rules of engagement.

Seniority level: Entry level

Employment type: Full-time

Industries: Technology, Information and Internet