Data Engineer with Automation Expertise

5 days ago


PuneRemote, India Cloudaeon Full time

About Cloudaeon: A Global Technology Consulting Company

Cloudaeon is a global technology consulting and services company that supports businesses in managing cloud infrastructure and solutions using big data, DevOps, and analytics.

We offer first-class solutions and services that utilize big data to exceed customer expectations.

Our deep vertical knowledge combined with expertise in several enterprise-class big data platforms helps us develop targeted solutions to meet our customers' business needs.

Our team consists of experienced professionals with experience in various tech stacks.

Skillful Senior Data Engineer with Automation Experience:

We are seeking a highly skilled Senior Data Engineer with expertise in automation testing and data engineering to support our test team in validating data pipelines and integration processes while also assisting the build team as required.

This role is critical in ensuring data quality, performance, and reliability in a large-scale enterprise environment, integrating data from SAP S/4HANA, BW/4HANA, and Kafka/DataSphere into an Azure Medallion Architecture and analytical platforms like Power BI.

Key Responsibilities:

  • Automation Testing & Data Validation
  • Automation Framework Development
  • Collaboration and Support

Automation Testing & Data Validation:

  1. Develop, automate, and maintain data validation workflows for seamless integration into Azure's Bronze, Silver, and Gold layers.
  2. Validate data pipelines for accuracy, completeness, and consistency across integration layers.
  3. Test incoming data from SAP S/4HANA, BW/4HANA, and Kafka/DataSphere for quality and adherence to standards.
  4. Conduct performance and scalability testing for large-scale data pipelines.
  5. Ensure data governance, compliance, and security standards are met throughout the testing lifecycle.

Automation Framework Development:

  1. Design and implement reusable automation frameworks for testing data pipelines and migrations in Azure.
  2. Develop automation scripts using SQL, Python, and tools like Databricks and Azure Data Services.
  3. Integrate automated testing processes into CI/CD pipelines to enable continuous deployment.

Collaboration and Support:

  1. Collaborate with the test manager and with the wider team to support test planning, defect tracking, and reporting.
  2. Assist the build team in troubleshooting and enhancing data pipelines or frameworks as needed.
  3. Provide technical guidance and mentorship to team members on data testing and pipeline automation.

Required Skills & Experience:

  1. 6+ years of experience in data engineering, with a focus on testing and automation for data migration and integration projects.
  2. Strong knowledge of Azure Medallion Architecture (Bronze, Silver, Gold layers).
  3. Proven experience validating data pipelines and optimizing their performance.

Must-Have Skills:

  • Databricks
  • SQL
  • Python
  • Azure Data Services (Data Factory, Synapse, Databricks)

Nice-to-Have Skills:

  • Kafka
  • TOSCA
  • Postman
  • SOAP UI

Experience with Power BI Integration and Reporting:

  • Familiarity with SAP S/4HANA and BW/4HANA data structures.
  • Strong problem-solving and analytical capabilities with effective communication skills.


  • Pune/Remote, India Cloudaeon Full time

    About Cloudaeon : Cloudaeon is a global technology consulting and services company. We support companies in managing cloud infrastructure and solutions with the help of big data, DevOps and analytics. We offer first-class solutions and services that use big data and always exceed customer expectations. Our deep vertical knowledge, combined with expertise in...

  • SR Data Engineer

    2 weeks ago


    Pune, Maharashtra, India Rockwell Automation Full time

    Rockwell Automation is a global technology leader focused on helping the world's manufacturers be more productive, sustainable, and agile. With more than 28,000 employees who make the world better every day, we know we have something special. Behind our customers - amazing companies that help feed the world, provide life-saving medicine on a global scale,...

  • SR Data Engineer

    23 hours ago


    Pune, India Rockwell Automation Full time

    Rockwell Automation is a global technology leader focused on helping the world’s manufacturers be more productive, sustainable, and agile. With more than 28,000 employees who make the world better every day, we know we have something special. Behind our customers - amazing companies that help feed the world, provide life-saving medicine on a global scale,...


  • Pune, Maharashtra, India Rockwell Automation Full time

    Rockwell Automation is a global technology leader focused on helping the worlds manufacturers be more productive sustainable and agile With more than 28 000 employees who make the world better every day we know we have something special Behind our customers - amazing companies that help feed the world provide life-saving medicine on a global scale and...

  • Senior Data Engineer

    3 weeks ago


    Pune/Remote, India Cloudaeon Full time

    About Cloudaeon : Cloudaeon is a global technology consulting and services company. We support companies in managing cloud infrastructure and solutions with the help of big data, DevOps and analytics. We offer first-class solutions and services that use big data and always exceed customer expectations. Our deep vertical knowledge, combined with expertise in...


  • Pune, Maharashtra, India Data Dynamics Full time

    QA Automation Engineer At Data Dynamics, we take a holistic approach to managing the explosive growth of unstructured data We care about delivering storage optimization and data optimization. Our Unified Unstructured Data Management Platform is an industry leader in Unstructured Data Management and our journey has just begun. With over 300 clients operating...


  • Pune, Maharashtra, India Data Dynamics Full time

    QA Automation Engineer At Data Dynamics, we take a holistic approach to managing the explosive growth of unstructured data We care about delivering storage optimization and data optimization. Our Unified Unstructured Data Management Platform is an industry leader in Unstructured Data Management and our journey has just begun. With over 300 clients operating...


  • Pune, Maharashtra, India Rockwell Automation Singapore Full time

    Job DescriptionRockwell Automation is a global technology leader focused on helping the world's manufacturers be more productive, sustainable, and agile. With more than 28,000 employees who make the world better every day, we know we have something special. Behind our customers - amazing companies that help feed the world, provide life-saving medicine on a...

  • Software Engineer

    17 hours ago


    Pune, Maharashtra, India SpurQLabs | Next-Generation Software Testing & Test Automation Services Full time

    About the JobAt SpurQLabs, we're seeking a skilled Python Developer to join our product development team. The successful candidate will focus on automating data ingestion, processing, and validation workflows using PySpark, ensuring robust, scalable, and efficient data pipelines.We're looking for an individual who can work closely with data engineers,...

  • Data Dynamics

    4 weeks ago


    Pune Division, India Data Dynamics Full time

    QA Automation Engineer At Data Dynamics, we take a holistic approach to managing the explosive growth of unstructured data! We care about delivering storage optimization and data optimization. Our Unified Unstructured Data Management Platform is an industry leader in Unstructured Data Management and our journey has just begun. With over 300 clients operating...


  • Pune/Remote, India Cloudaeon Full time

    About Cloudaeon: Innovative Cloud Solutions ProviderCloudaeon is a global technology consulting company that offers innovative cloud solutions and services to support organizations in managing cloud infrastructure and solutions using big data, DevOps, and analytics.We have a deep understanding of enterprise-class big data platforms and use this expertise to...


  • Pune, Maharashtra, India Rockwell Automation Full time

    Rockwell Automation is a global technology leader focused on helping the worldxe2x80x99s manufacturers be more productive sustainable and agile With more than 28 000 employees who make the world better every day we know we have something special Behind our customers - amazing companies that help feed the world provide life-saving medicine on a global...


  • Pune, Maharashtra, India TransPerfect Full time

    ResponsibilitiesAs a Senior IT Systems Engineer at TransPerfect, your responsibilities will include:1. Configuring, setting up, maintaining, and troubleshooting servers and services - automation of redundant processes2. Participating in infrastructure and capacity planning3. Participating in the planning, designing, and testing of disaster recovery...


  • Pune, India Rockwell Automation Singapore Full time

    Job Description Rockwell Automation is a global technology leader focused on helping the world's manufacturers be more productive, sustainable, and agile. With more than 28,000 employees who make the world better every day, we know we have something special. Behind our customers - amazing companies that help feed the world, provide life-saving medicine on a...


  • Pune, India Rockwell Automation Full time

    Rockwell Automation is a global technology leader focused on helping the world’s manufacturers be more productive, sustainable, and agile. With more than 25,000 employees who make the world better every day, we know we have something special. Behind our customers - amazing companies that help feed the world, provide life-saving medicine on a global scale,...

  • Cubix Automation

    4 weeks ago


    Pune Division, India Cubix Automation Full time

    Role : PLC ProgrammerCompany Description :Cubix Automation Private Limited (CAPL) is a leading provider of consulting services for advanced control systems and factory automation solutions based in Pune, India. Specializing in end-to-end solutions-from concept to commissioning-CAPL serves a diverse clientele including end-users, original equipment...


  • Pune, Maharashtra, India Rockwell Automation Full time

    Role - Java Full stack developerReport to ManagerSenior Software Engineer – Full Stack Java DeveloperRockwell is looking for an all-rounder Software Engineer, Project Level, to join an agile team building the next generation of analytics for Industrial Automation. We are looking for someone who is well versed in Agile practices, is customer focused, has...

  • Proposal Engineer

    3 weeks ago


    Pune, Maharashtra, India Rockwell Automation Full time

    Rockwell Automation is a global technology leader focused on helping the worldxe2x80x99s manufacturers be more productive sustainable and agile With more than 28 000 employees who make the world better every day we know we have something special Behind our customers - amazing companies that help feed the world provide life-saving medicine on a global...


  • Pune, Maharashtra, India NTT DATA Full time

    Job Description Req ID: 317412 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Azure Cloud Automation Consultant to join our team in pune, Mahārāshtra (IN-MH), India...


  • Pune, India NTT DATA Full time

    Job Description Req ID: 317412 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Azure Cloud Automation Consultant to join our team in pune, Mahārāshtra (IN-MH), India...