Define and report key performance indicators and launch impact as part of regular business reviews with the cross-functional and cross-organizational leadership team. Translate analysis results to business insights or product improvement opportunities.
Develop hypothesis to enhance performance of AI products on offline and online metrics through research on techniques around prompt engineering, RAG, supervised finetuning, in-context learning, dataset augmentation, tool-calling efficacy, planning capabilities and feedback loop with reinforcement learning.
Design and develop ML strategies for data enrichment such as autoencoder based latent variables, complex heuristics etc.
Evolve variance reduction and simulation strategies to increase reliability of experiments with small sample sizes. Unlock continually improving experimentation with algorithms like contextual bandits.
Convert business problems into unsupervised and supervised machine learning modeling problems, and build these model prototypes from scratch to justify business impact hypothesis.
Minimum qualifications:
Bachelor's degree in Statistics, Mathematics, Data Science, Engineering, Physics, Economics, or a related quantitative field.
10 years of experience using analytics to solve product or business problems, performing statistical analysis, and coding (e.g., Python, R, SQL) or 8 years of experience with a Master's degree.
Preferred qualifications:
Experience with developing at least one deep learning or conventional machine learning model for business impact.
Experience debugging throughput, latency and response quality issues in AI products, from an analytical perspective.
Experience managing large-scale data transformation pipelines for batch inference of ML models.
