Data Integration Development
Design and implement data integration pipelines connecting multiple data sources
Build data transformation logic and mapping between different data formats
Create and maintain database schemas, tables, and data models
Optimize data pipeline performance and troubleshoot integration issues
Develop ETL/ELT processes for real-time and batch data processing
Database Operations & Management
Manage database operations across multiple platforms (PostgreSQL, MongoDB, ClickHouse, DuckDB, Snowflake
Perform database administration tasks including performance tuning and optimization
Design efficient database structures and indexing strategies
Monitor database health, connectivity, and performance metrics
Handle database migrations and schema evolution
Platform Operations & Automation
Maintain Kafka topics and streaming data flows (training provided)
Automate deployment and configuration processes using Terraform
Monitor integration pipeline health and resolve operational issues
Implement SRE practices for reliable data platform operations
Document integration processes and troubleshooting procedures
Required Skills
Must have minimum 6+ years of relevant experience
Strong experience in stream processing using Apache Flink and messaging systems like Apache Kafka
Hands-on experience in building real-time and batch ETL/ELT pipelines and data integration workflows
Proficiency in Python / Java / Scala with strong data transformation and processing skills
Experience with databases such as PostgreSQL, MongoDB, ClickHouse, DuckDB, and Snowflake
Strong understanding of data modeling, indexing, and performance tuning
Experience with pipeline monitoring, troubleshooting, and optimization
Knowledge of Terraform and CI/CD for automation and deployment
Understanding of SRE practices and platform reliability
Good problem-solving, debugging, and documentation skills
