Data Automation Engineer

3 days ago

Pune, Maharashtra, India beBeeData Full time
Job Title: Data Automation Engineer

At the forefront of modern data processing lies Apache DolphinScheduler, a powerful tool for automating workflows and streamlining data pipelines. As a seasoned Data Automation Engineer, you will play a pivotal role in harnessing the full potential of this technology.

  • Key Responsibilities:
  • Develop, configure, and optimize Apache DolphinScheduler workflows for efficient data processing and automation.
  • Design and implement ETL pipelines, job scheduling, and workflow orchestration solutions to drive business growth.
  • Troubleshoot and resolve performance, scalability, and reliability issues in DolphinScheduler to ensure seamless operations.
  • Integrate DolphinScheduler with big data tools such as Hadoop, Spark, Flink, Hive, and Kafka to expand its capabilities.
  • Collaborate closely with data engineers, DevOps, and software teams to enhance workflow automation and drive innovation.
  • Develop custom plugins and extensions for DolphinScheduler as needed to meet evolving business needs.
  • Monitor and optimize job execution, resource allocation, and workflow efficiency to maximize ROI.
  • Adhere to best practices for CI/CD, version control, and infrastructure automation to ensure quality and reliability.
Required Skills and Qualifications

To excel in this role, you will require:

  • Proven experience with Apache DolphinScheduler in a production environment.
  • Proficiency in Java, Python, or Scala for workflow scripting and automation.
  • Hands-on experience with big data technologies like Hadoop, Spark, Hive, Flink, and Kafka.
  • Strong understanding of workflow orchestration, DAG execution, and scheduling strategies.
  • Excellent problem-solving skills and ability to thrive in a fast-paced environment.
Benefits

As a valued member of our team, you can expect:

  • A dynamic and supportive work environment that fosters growth and collaboration.
  • The opportunity to work on cutting-edge projects and technologies.
  • Ongoing training and professional development to stay ahead of the curve.
  • A competitive compensation package and benefits.

  • Quality Engineer

    4 days ago

    Pune, Maharashtra, India Rockwell Automation Full time US$ 90,000 - US$ 1,20,000 per year

    Rockwell Automation is a global technology leader focused on helping the world's manufacturers be more productive, sustainable, and agile. With more than 28,000 employees who make the world better every day, we know we have something special. Behind our customers - amazing companies that help feed the world, provide life-saving medicine on a global scale,...

  • QA Automation

    2 weeks ago

    Pune, Maharashtra, India NTT Data Full time

    Job DescriptionReq ID:327777NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a QA Automation - Digital Solution Cnslt. Sr. Consultant to join our team in Pune, Mahrshtra (IN-MH), India...

  • Senior Data Automation Engineer

    11 hours ago

    Pune, Maharashtra, India beBeeAutomation Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job OverviewWe are seeking a Data Automation Specialist to develop and maintain data validation frameworks. The successful candidate will ensure stability of data pipelines and metadata management.Key Responsibilities:Develop and maintain data validation frameworks.Ensure stability of data pipelines and metadata management.Provide technical support and...

  • Project Engineer-pharma

    4 weeks ago

    Pune, Maharashtra, India Rockwell Automation Full time

    Rockwell Automation is a global technology leader focused on helping the world s manufacturers be more productive sustainable and agile With more than 28 000 employees who make the world better every day we know we have something special Behind our customers - amazing companies that help feed the world provide life-saving medicine on a global scale ...

  • Senior QE Engineer

    2 weeks ago

    Pune, Maharashtra, India NTT Data Full time

    Job DescriptionReq ID:330977NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior QE Engineer to join our team in Pune, Mahrshtra (IN-MH), India (IN).Senior Quality Engineer I...

  • Senior QE Engineer

    21 hours ago

    Pune, Maharashtra, India NTT Data Full time

    Job DescriptionReq ID:330974NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior QE Engineer to join our team in Pune, Mahrshtra (IN-MH), India (IN).Senior Quality Engineer I...

  • Business Analyst-data Analytics

    4 weeks ago

    Pune, Maharashtra, India Rockwell Automation Full time

    Rockwell Automation is a global technology leader focused on helping the worlds manufacturers be more productive sustainable and agile With more than 28 000 employees who make the world better every day we know we have something special Behind our customers - amazing companies that help feed the world provide life-saving medicine on a global scale and...

  • Quality Assurance Engineer

    4 days ago

    Pune, Maharashtra, India Rockwell Automation Full time US$ 90,000 - US$ 1,20,000 per year

    Rockwell Automation is a global technology leader focused on helping the world's manufacturers be more productive, sustainable, and agile. With more than 28,000 employees who make the world better every day, we know we have something special. Behind our customers - amazing companies that help feed the world, provide life-saving medicine on a global scale,...

  • Project Engineer-Pharma

    4 days ago

    Pune, Maharashtra, India Rockwell Automation Full time US$ 80,000 - US$ 1,20,000 per year

    Rockwell Automation is a global technology leader focused on helping the world's manufacturers be more productive, sustainable, and agile. With more than 28,000 employees who make the world better every day, we know we have something special. Behind our customers - amazing companies that help feed the world, provide life-saving medicine on a global scale,...

  • Automation Engineer

    2 days ago

    Pune, Maharashtra, India Tetra Pak Full time

    At Tetra Pak we commit to making food safe and available everywhere and we protect what s good - protecting food protecting people and protecting the planet By doing so we touch millions of people s lives every day And we need people like you to make it happen We empower you to reach your potential with opportunities to make an impact to be proud of...