Highly Skilled Data Engineer Wanted

Senior Data Engineer II

We are seeking a highly skilled and experienced data engineer to join our dynamic team in Bengaluru. This is an exciting opportunity to contribute to a data-driven culture and directly impact our analytical capabilities.

Key Responsibilities :

1. ">
2. Data Pipeline Development : Design, develop, and optimize scalable ETL/ELT data pipelines using AWS services and Python to ingest, process, and transform diverse datasets from various sources.">
3. AWS Cloud Infrastructure : Build and manage data infrastructure components within the AWS ecosystem, including but not limited to Amazon S3, AWS Glue, AWS Lambda, Amazon Redshift/Snowflake, and other relevant data services.">
4. Data Modeling & Warehousing : Develop and maintain optimized data models for our data warehouse, ensuring data integrity, consistency, and performance for analytical and reporting needs.">
5. Performance Optimization : Identify and implement solutions for improving data pipeline efficiency, reliability, and cost-effectiveness.">
6. Data Quality & Governance : Establish and enforce data quality standards, implement monitoring, and develop mechanisms for data validation and error handling.">
7. Collaboration & Mentorship : Work closely with data scientists, analysts, and product teams to understand data requirements and deliver solutions. Provide technical guidance and mentorship to junior engineers.">
8. Troubleshooting & Support : Troubleshoot and resolve complex data-related issues, ensuring minimal disruption to data availability and accuracy.">
9. Documentation : Create and maintain comprehensive documentation for data pipelines, architecture, and processes.">

Required Qualifications :

• ">
• Experience : Minimum of 5-10 years of progressive experience in data engineering, backend engineering with a strong data focus, or a related field.">
• AWS Expertise : Strong proficiency and hands-on experience in building and maintaining data solutions within the AWS cloud environment. This includes deep familiarity with services such as Storage : Amazon S3 Compute/Orchestration : AWS Glue, AWS Lambda, AWS Step Functions, EC2 Data Warehousing : Amazon Redshift (or similar cloud data warehouses like Snowflake, Google BigQuery) Networking/Security : VPC, IAM">
• Python Proficiency : Advanced proficiency in Python for data manipulation, scripting, automation, and developing data processing applications. Experience with libraries like Pandas, PySpark, or similar is a plus.">
• Advanced SQL Knowledge : Expert-level SQL skills for complex querying, data manipulation (DDL/DML), performance tuning, and optimizing database interactions.">
• Data Warehousing Concepts : Solid understanding of data warehousing principles, dimensional modeling (Kimball/Inmon), and star/snowflake schemas.">
• Pipeline Orchestration : Experience with workflow orchestration tools (e.g., Apache Airflow, AWS Step Functions) for scheduling and monitoring data pipelines.">
• Version Control : Proficient with Git and collaborative development workflows.">
• Education : Bachelors degree in Computer Science, Engineering, Information Technology, or a related quantitative field is preferred; equivalent practical experience and demonstrated expertise are highly valued.">

Preferred Qualifications :

• ">
• Experience with real-time data processing and streaming technologies (e.g., Kinesis, Kafka).">
• Familiarity with containerization technologies (Docker, Kubernetes).">
• Experience with CI/CD practices for data pipelines.">
• Knowledge of big data frameworks (e.g., Apache Spark).">
• AWS Certifications (e.g., AWS Certified Data Analytics Specialty, AWS Certified Solutions Architect Associate)">