Come work at a place where innovation and teamwork come together to support the most exciting missions in the world!
Job Description
We are seeking a Data Science focused QA engineer to develop next-generation Security Analytics products. You will work closely with Data scientists,
engineers
and product managers to design and
optimize
AI driven security solutions.
As
QA
engineer, the ideal candidate has a strong background in Backend engineering, system integrations,
ML,AI
and data pipelines.
Responsibilities (QA Engineer – Data Science / ML)
Establish QA best practices for Traditional ML and Generative AI workflows, including:
Functional and regression testing of ML pipelines using
pytest
and Airflow/
Dagster
test utilities
and API testing tools (e.g., Postman,
pytest-httpx
).
Validate data contracts, schemas, and API compatibility across services using
Pandera
, and custom validation rules
.
Model behavior validation (input/output ranges, invariants, edge cases) using NumPy, SciPy, and statistical assertions
Runtime and performance testing for inference latency, throughput, and resource usage using Locust, k6, or custom load tests
.
Integrate ML-specific tests into CI/CD pipelines using GitHub Actions, GitLab CI, or Jenkins, alongside containerized workflows (Docker, Kubernetes).
Implement LLM-specific testing, including:
Prompt and response validation, determinism checks, and regression testing using
LangSmith
.
Evaluation of hallucinations, toxicity, and policy adherence using LLM-as-a-judge and
/or
rule-based checks
.
Cost, token usage, and timeout monitoring for GenAI workflows
Verify logging, monitoring, and alerting for ML services using Prometheus, Grafana, and cloud-native observability tools.
Requirements:
BS or MS
in
Computer Science or a related field
.
2-5 years
of experience in Data
or Machine
Learning
projects
.
Familiarity and
experience
of GenAI applications
and tools -
PyTorch
,
LangChain
,
vLLM
etc.
Demonstrates
a commitment to continuous learning in this rapidly evolving field.
Tools listed in
the responsibilities
section.
