About Us
Visa is a world leader in payments technology, facilitating transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories, dedicated to uplifting everyone, everywhere by being the best way to pay and be paid.
At Visa, you'll have the opportunity to create impact at scale — tackling meaningful challenges, growing your skills and seeing your contributions impact lives around the world.
Join Visa and do work that matters – to you, to your community, and to the world. Progress starts with you.
Job Description
At Visa, the Corporate Information Technology, Billing & Incentives Platforms team, enables Visa's revenue growth through flexible rules-based pricing engines and global revenue applications built on next-generation technologies. This includes managing system requirements, evaluating cutting-edge technologies, design, development, integration, quality assurance, implementation, and maintenance of corporate revenue applications. The team works closely with business owners of these services to deliver custom developed solutions, as well as implement industry leading packaged software. This team has embarked on a major transformational journey to build and implement best of breed revenue and billing applications to transform our business as well as technology.
We are looking for a Senior Software Engineer who will work in Core Billing Platform with strong hands-on experience in PySpark, Spark SQL, and Scala to design, develop, and optimize scalable data pipelines for large-scale data processing.
The ideal candidate should have deep expertise in distributed data processing, ETL development, performance tuning, and big data ecosystem technologies. This role requires end-to-end ownership of data pipelines, from ingestion and transformation to storage and consumption, while ensuring high performance, reliability, and data quality.
Exposure to AI/GenAI enabled data engineering solutions is a plus.
Key Responsibilities:
Design, develop, and maintain scalable data pipelines using PySpark, Spark SQL, and Scala
Build and optimize complex ETL workflows for extracting, transforming, and loading data across multiple systems such as Oracle, PostgreSQL, Hive, Hadoop, and cloud-based data platforms
Develop and optimize Spark jobs for large-scale batch and, where applicable, streaming data processing
Write efficient Spark SQL queries and optimize data transformations for performance and scalability
Implement data processing strategies including partitioning, caching, parallel processing, file format optimization, and job tuning
Architect and implement big data solutions using Spark, Hive, Hadoop ecosystem, Delta Lake, and related technologies
Build end-to-end data flows from data ingestion → transformation → storage → consumption layers
Develop config-driven, reusable ETL frameworks, automation utilities, and common data engineering components
Ensure data quality, integrity, accuracy, and consistency across data pipelines
Perform performance tuning, debugging, troubleshooting, and root-cause analysis of data pipeline issues
Collaborate with business stakeholders, data analysts, architects, and engineering teams to translate business requirements into scalable technical solutions
Create and maintain technical documentation, data flow diagrams, data models, and architecture artifacts
Follow engineering best practices around code quality, version control, CI/CD, testing, deployment, monitoring, and alerting
Mentor junior engineers and promote best practices in data engineering and big data development
Explore and integrate AI/GenAI capabilities where applicable for automation, data validation, anomaly detection, and ETL optimization
This is a hybrid position. Expectation of days in office will be confirmed by your hiring manager.
Qualifications
Qualifications:
Bachelor’s or master’s degree in Computer Science, Software Engineering or other relevant Engineering discipline with 3-5 years of experience in Data Engineering, Big Data Engineering, or ETL development
Strong hands-on experience with PySpark, Spark SQL, and Scala
Solid experience building and maintaining large-scale ETL/ELT data pipelines
Strong understanding of Apache Spark architecture, including executors, drivers, DAGs, stages, tasks, shuffle, caching, and memory management
Proven experience in Spark performance tuning and optimization
Strong SQL skills with experience writing complex queries, joins, aggregations, window functions, and query optimization
Experience working with big data technologies such as Hive, Hadoop, HDFS, Delta Lake, Parquet, ORC, and related ecosystem tools
Experience working with relational databases such as Oracle, PostgreSQL, SQL Server, or MySQL
Strong programming skills in Python and Scala
Good understanding of distributed computing, data partitioning, data modeling, and large-scale data processing patterns
Experience with data quality checks, reconciliation, validation, exception handling, and audit controls
Familiarity with CI/CD pipelines, Git/version control, code reviews, and deployment processes
Experience working in Agile/Scrum delivery environments
Strong problem-solving, analytical, and communication skill
AI / GenAI Exposure — Good to Have:
Exposure to AI/ML or Generative AI concepts
Experience integrating AI/LLM APIs into data pipelines, applications, or automation workflows
Understanding of data pipelines supporting ML/AI model development and deployment
Experience using AI tools for:
ETL code generation
Data validation
Anomaly detection
Data profiling
Workflow automation
Ability to identify opportunities to improve data engineering productivity using AI-assisted development tools
Visa is an EEO Employer
Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.
