Job Role: Senior Software Engineer
Location: Hyderabad
Experience : 5 Years
About the Role
We are seeking a Senior Software Engineer to design and build large-scale identity resolution and graph-based data systems. This role will focus on constructing and optimizing distributed data pipelines that power an identity graph used for entity resolution, cross-device matching, and data enrichment.
You will work at the intersection of big data processing, graph modeling, and distributed systems, building high-performance pipelines that process billions of records across cloud infrastructure.
This is a hands-on engineering role requiring deep experience in Spark-based distributed systems and cloud-native data platforms.
What You’ll Do
Design and implement large-scale identity resolution pipelines using Spark
Develop high-performance data processing services in Scala, Python and Java
Build and maintain graph-based identity models and
Senior Software Engineer – Identity Resolution
About the Role
We are seeking a Senior Software Engineer to design and build large-scale identity resolution and graph-based data systems. This role will focus on constructing and optimizing distributed data pipelines that power an identity graph used for entity resolution, cross-device matching, and data enrichment.
You will work at the intersection of big data processing, graph modeling, and distributed systems, building high-performance pipelines that process billions of records across cloud infrastructure.
This is a hands-on engineering role requiring deep experience in Spark-based distributed systems and cloud-native data platforms.
What You’ll Do
Design and implement large-scale identity resolution pipelines using Spark
Develop high-performance data processing services in Scala, Python and Java
Build and maintain graph-based identity models and entity resolution workflows
Optimize distributed workloads on Databricks
Architect scalable matching, clustering, and weak-edge detection algorithms
Improve observability and performance of data processing jobs
Collaborate with data scientists and product teams to operationalize graph models
Enforce best practices in data quality, lineage, and governance
Required Skills & Experience
Strong hands-on experience with Apache Spark
Proficiency in Scala, Python and Java
Experience deploying and managing workloads in AWS
Experience building and optimizing jobs in Databricks
Deep understanding of distributed systems and large-scale data processing
Experience designing scalable ETL/ELT pipelines
Strong SQL skills and understanding of data modeling
Familiarity with cloud-native architectures (AWS preferred)
Nice to Have
Experience building or working with identity graphs
Experience with identity resolution / entity resolution systems
Knowledge of graph algorithms (transitive closure, connected components, clustering, matching)
Experience working with large-scale Parquet datasets
Exposure to graph databases (e.g., Neo4j, Neptune) is a plus
Understanding of privacy-safe data linking techniques
