Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field.
3-4 years of hands-on experience in data engineering, with a strong focus on AWS cloud services.
Proficiency in Python for data manipulation, scripting, and automation.
Strong command of SQL for data querying, transformation, and database management.
Demonstrable experience with AWS data services, including:
Amazon S3: Data Lake storage and management.
AWS Glue: ETL service for data preparation.
Amazon Redshift: Cloud data warehousing.
AWS Lambda: Serverless computing for data processing.
Amazon EMR: Managed Hadoop framework for big data processing (Spark/PySpark experience highly preferred).
AWS Kinesis (or Kafka): Real-time data streaming.
Strong analytical, problem-solving, and debugging skills.
Excellent communication and collaboration abilities, with the capacity to work effectively in an agile team environment.
Responsibilities:
Troubleshoot and resolve data-related issues and performance bottlenecks in existing pipelines.
Develop and maintain data quality checks, monitoring, and alerting mechanisms to ensure data pipeline reliability.
Participate in code reviews, contribute to architectural discussions, and promote best practices in data engineering.
