Data Platform Engineer

12 hours ago

Bengaluru, India Epergne Solutions Full time


Epergne Solutions is looking for Data Platform Engineer (Python + Airflow Data Engineer)


Total Yrs. of Experience* 6+ yrs

Relevant Yrs. of experience* 5+ yrs

Mandatory skills* Python / Airflow Data Engineer

Budget: As per market Standards

Work Location* India

Mode- Work from Office

Positions: Contract

Payroll: Epergne solutions


Detailed JD *(Roles and Responsibilities )


We are seeking a highly skilled with 3 to 6 years of experience, specifically with a strong background in AWS technologies. The ideal

candidate will have a deep understanding of Apache Airflow and its integration within AWS ecosystem, enabling efficient data pipeline orchestration and management.


Responsibilities:


  • Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration.
  • Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.
  • Implement data integration and transformation processes to ensure optimal performance and reliability of data pipelines.
  • Optimize and fine-tune existing data pipelines / Airflow to improve efficiency, scalability, and maintainability.
  • Troubleshoot and resolve issues related to data pipelines, ensuring smooth operation and minimal downtime.
  • Work closely with AWS services like S3, Glue, EMR, Redshift, and other related technologies to design and optimize data infrastructure.
  • Develop and maintain documentation for data pipelines, processes, and system architecture.
  • Stay updated with the latest industry trends and best practices related to data engineering and AWS services.


Requirements:

  • Bachelor s degree in Computer Science, Engineering, or a related field.
  • Proficiency in Python and SQL for data processing and manipulation.
  • Min 5 years of experience in data engineering, specifically working with Apache Airflow and AWS technologies.
  • Strong knowledge of AWS services, particularly S3, Glue, EMR, Redshift, and AWS Lambda.
  • Understanding of Snowflake is preferred.


  • Experience with optimizing and scaling data pipelines for performance and efficiency.
  • Good understanding of data modeling, ETL processes, and data warehousing concepts.
  • Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment.
  • Effective communication skills and the ability to articulate technical concepts to non-technical stakeholders.


Preferred Qualifications:

  • AWS certification(s) related to data engineering o r big data.
  • Experience working with big data technologies like Snowflake, Spark, Hadoop, or related frameworks.
  • Familiarity with other data orchestration tools in addition to Apache Airflow.
  • Knowledge of version control systems like Bitbucket, Git


Preferred candidates who can join in 30 days/lesser Notice Period

  • Python Data Platform Engineer

    10 hours ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID:     NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Python Data Platform Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Job...

  • Python Data Platform Engineer

    7 hours ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID:     NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Python Data Platform Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN). ...

  • Senior Automation Platform Engineer

    2 weeks ago

    Bengaluru, India NTT DATA Full time

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Senior Automation Platform Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Job Summary: The...

  • It Data Platform Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Palo Alto Networks Full time

    Company Description Our Mission At Palo Alto Networks everything starts and ends with our mission Being the cybersecurity partner of choice protecting our digital way of life Our vision is a world where each day is safer and more secure than the one before We are a company built on the foundation of challenging and disrupting the way things are done ...

  • IT Data Platform Engineer

    2 weeks ago

    Bengaluru, India Palo Alto Networks Full time

    Our Mission At Palo Alto Networks® everything starts and ends with our mission: Being the cybersecurity partner of choice, protecting our digital way of life. Our vision is a world where each day is safer and more secure than the one before. We are a company built on the foundation of challenging and disrupting the way things are done, and we’re...

  • IT Data Platform Engineer

    7 days ago

    Bengaluru, India Palo Alto Networks Full time

    Our MissionAt Palo Alto Networks® everything starts and ends with our mission:Being the cybersecurity partner of choice, protecting our digital way of life.Our vision is a world where each day is safer and more secure than the one before. We are a company built on the foundation of challenging and disrupting the way things are done, and we’re looking for...

  • Platform Support Engineer

    12 hours ago

    Bengaluru, India NTT DATA Full time

    Job Description Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive. Your day at...

  • Platform Support Engineer

    7 hours ago

    Bengaluru, India NTT DATA Full time

    Job Description Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive. Your day...

  • Data Platform Engineer

    4 days ago

    Bengaluru, Karnataka, India Palo Alto Networks Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    IT Data Platform Engineer Company Description Our Mission At Palo Alto Networks everything starts and ends with our mission: Being the cybersecurity partner of choice, protecting our digital way of life. Our vision is a world where each day is safer and more secure than the one before. We are a company built on the foundation of challenging and...

  • IT Data Platform Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Palo Alto Networks Full time

    Our MissionAt Palo Alto Networks everything starts and ends with our mission:Being the cybersecurity partner of choice, protecting our digital way of life.Our vision is a world where each day is safer and more secure than the one before. We are a company built on the foundation of challenging and disrupting the way things are done, and we're looking for...