Cloud Data Engineering Leader

6 days ago


PuneRemote, India Cloudaeon Full time

About Cloudaeon: Innovative Cloud Solutions Provider

Cloudaeon is a global technology consulting company that offers innovative cloud solutions and services to support organizations in managing cloud infrastructure and solutions using big data, DevOps, and analytics.

We have a deep understanding of enterprise-class big data platforms and use this expertise to create targeted solutions that meet our customers' business needs.

We're currently seeking a skilled Senior Data Engineer with Automation Experience:

We're looking for a senior data engineer with expertise in automation testing and data engineering to support our test team in validating data pipelines and integration processes while also assisting the build team as required.

This role is critical in ensuring data quality, performance, and reliability in a large-scale enterprise environment, integrating data from SAP S/4HANA, BW/4HANA, and Kafka/DataSphere into an Azure Medallion Architecture and analytical platforms like Power BI.

Key Job Responsibilities:

  • Data Engineering and Automation
  • Azure Medallion Architecture Implementation
  • Cross-Functional Team Collaboration

Data Engineering and Automation:

  1. Develop, automate, and maintain data validation workflows for seamless integration into Azure's Bronze, Silver, and Gold layers.
  2. Validate data pipelines for accuracy, completeness, and consistency across integration layers.
  3. Test incoming data from SAP S/4HANA, BW/4HANA, and Kafka/DataSphere for quality and adherence to standards.
  4. Conduct performance and scalability testing for large-scale data pipelines.
  5. Ensure data governance, compliance, and security standards are met throughout the testing lifecycle.

Azure Medallion Architecture Implementation:

  1. Implement reusable automation frameworks for testing data pipelines and migrations in Azure.
  2. Develop automation scripts using SQL, Python, and tools like Databricks and Azure Data Services.
  3. Integrate automated testing processes into CI/CD pipelines to enable continuous deployment.

Cross-Functional Team Collaboration:

  1. Collaborate with the test manager and with the wider team to support test planning, defect tracking, and reporting.
  2. Assist the build team in troubleshooting and enhancing data pipelines or frameworks as needed.
  3. Provide technical guidance and mentorship to team members on data testing and pipeline automation.

Requirements:

  1. 6+ years of experience in data engineering, with a focus on testing and automation for data migration and integration projects.
  2. Strong knowledge of Azure Medallion Architecture (Bronze, Silver, Gold layers).
  3. Proven experience validating data pipelines and optimizing their performance.

Mandatory Skills:

  • Databricks
  • SQL
  • Python
  • Azure Data Services (Data Factory, Synapse, Databricks)

Desirable Skills:

  • Kafka
  • TOSCA
  • Postman
  • SOAP UI

Power BI Integration Experience:

  • Familiarity with SAP S/4HANA and BW/4HANA data structures.
  • Strong problem-solving and analytical capabilities with effective communication skills.
],

  • Pune, Maharashtra, India Allianz Services Full time

    About the Role:We are seeking an experienced Senior Cloud Data Architect to join our team at Allianz. As a key member of our data architecture team, you will be responsible for designing and implementing scalable data products using cutting-edge technologies.Key Responsibilities:Data Product Design and Implementation: Collaborate with agile and analytics...


  • Pune, Maharashtra, India NTT DATA Full time

    Role Title Cloud or Platform EngineerSkills Description Qualification Bachelor s or Master s degree in Computer Science Engineering or a related field For an Engineer level position a minimum of 5 years of IT experience in software development is essential Candidates for Senior Engineer level must have a minimum of 8 years of IT experience specifically...


  • Pune, Maharashtra, India Capgemini Engineering Full time

    Position Name: Engineering leader for Industrial Automation Position Overview: The Engineering Leader will be responsible for driving innovation, strategy, and execution within the Industrial Automation accounts. The individual will lead cross-functional teams to design and deliver cutting-edge automation solutions, foster technology adoption across...


  • Pune, Maharashtra, India ACL Digital Full time

    About the Role:A high-profile opportunity has emerged for a seasoned ETL Data Architect to spearhead our data engineering efforts at ACL Digital. This is a pivotal moment in the company's journey, and we require a visionary leader with extensive experience in designing and implementing large-scale data architectures.Responsibilities:- Spearheading...


  • Pune, Maharashtra, India Capgemini Engineering Full time

    Position Name: Engineering leader for Industrial AutomationPosition Overview:The Engineering Leader will be responsible for driving innovation, strategy, and execution within the Industrial Automation accounts. The individual will lead cross-functional teams to design and deliver cutting-edge automation solutions, foster technology adoption across industries...

  • Senior Data Engineer

    2 weeks ago


    Pune, Maharashtra, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...


  • Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...


  • Remote, India Fractal31 Pvt Ltd Full time

    About the RoleFbral is looking for an experienced Data Engineer to lead our cloud migration efforts. As a key member of our data engineering team, you will be responsible for designing and implementing large-scale data systems, as well as collaborating with cross-functional teams to drive business outcomes.The ideal candidate will have expertise in...


  • Remote, India Cloud Destinations Full time

    About the Role :- We are seeking a highly skilled and experienced Oracle Cloud Administrator to join our dynamic team.- In this role, you will be responsible for the administration and maintenance of our Oracle Cloud Infrastructure (OCI) and Oracle Integration Cloud (OIC) environments.- You will play a critical role in ensuring the availability, performance,...


  • Remote, India Cloud Destinations Full time

    About the Position:At Cloud Destinations, we are looking for a Cloud Computing Administrator with expertise in Oracle Cloud Infrastructure (OCI) and Oracle Integration Cloud (OIC).The ideal candidate will have hands-on experience with OCI services, including compute, storage, networking, security, and integration.Responsibilities:Administer and maintain OCI...


  • Remote, India Cloud Destinations Full time

    About the Role:We are seeking a highly skilled and experienced Oracle Cloud Administrator to join our dynamic team.As an Oracle Cloud Solutions Engineer at Cloud Destinations, you will play a critical role in ensuring the availability, performance, and security of our cloud-based applications and services.Key Responsibilities:Manage and maintain OCI and OIC...

  • Data Scientist

    3 days ago


    Pune, India Change Leader Full time

    **Skills & Experience**: - Programming Skills - knowledge of statistical programming languages like R, Python, and database query languages like SQL, Hive, Pig is desirable. Familiarity with Scala, Java, or C++ is an added advantage. - Statistics - Good applied statistical skills, including knowledge of statistical tests, distributions, regression, maximum...


  • Remote, India FANISKO PRIVATE LIMITED Full time

    Data Engineering ExpertiseFANISKO PRIVATE LIMITED seeks a skilled Data Engineer to design and build scalable data pipelines using various tools and technologies. As a Data Engineer, you will be responsible for developing and managing cloud-based data storage solutions and databases.Responsibilities:Design and implement efficient data pipelines for processing...


  • Pune, Maharashtra, India Impetus Career Consultants Full time

    About Impetus Career Consultants We empower providers, payers and patients by enhancing financial performance and advancing the healthcare economy through our leading RCM solutions, intelligent automation, seamless integrations and data insights. About the Role: This leadership role is responsible for guiding a team of experienced data engineers in...

  • Data Engineer

    4 weeks ago


    Remote, India Mig Staffing Full time

    We are seeking a dynamic and highly skilled Data Engineer who has extensive experience building enterprise scale data platforms and lead these foundational efforts. This role demands someone who not only possesses a profound understanding of the data engineering landscape but is also at the forefront of their game. The ideal candidate will contribute...


  • Remote, India Coders Brain Technology Private Limited Full time

    **Job Overview**Coders Brain Technology Private Limited is a leading technology firm that specializes in providing innovative data analytics solutions to various industries. We are currently seeking an experienced Cloud Data Engineer to join our team.As a Cloud Data Engineer, you will be responsible for designing, developing, and maintaining interactive...


  • Pune, Maharashtra, India IBM Full time

    IBM Consulting is a leader in the field of hybrid cloud and AI. As a Data Engineer, you will work with our clients across various industries to improve their cloud journey. Your role will involve designing and implementing data management systems, identifying critical data for insightful analysis, and tackling challenges related to database integration.You...


  • Remote, India Cloud Destinations Full time

    About the Role :- We are seeking a highly skilled and experienced Oracle Cloud Administrator to join our dynamic team.- In this role, you will be responsible for the administration and maintenance of our Oracle Cloud Infrastructure (OCI) and Oracle Integration Cloud (OIC) environments.- You will play a critical role in ensuring the availability, performance,...


  • Pune, Maharashtra, India Genpact Full time

    Genpact is a global professional services firm driven by curiosity, entrepreneurial agility, and a desire to create lasting value for clients.Job DescriptionWe are inviting applications for the role of Lead Consultant-Data Engineer, AWS+Python. The successful candidate will be responsible for designing and deploying scalable, highly available, and...

  • Data Pm- Scrum

    3 days ago


    Pune, India Change Leader Full time

    **Desired Skills & Responsibilities **Primary Skill**:Data Engineering Project Management **Secondary Skill**:Agile Scrum Master 5-10 Years of Project Management experience Experience in Managing Data Engineering Projects Experience in Managing Data Transformation, Bigdata and Data Migration Projects Development and Testing experience in ETL/ELT...