
Data Pipeline Engineer
1 week ago
We are seeking a seasoned Data Pipeline Engineer with expertise in advanced testing to join our team for an exciting project.
About the Role:
The successful candidate will be responsible for performing complex data pipeline testing, developing and executing SQL queries for data validation, reconciliation, and transformation checks, and building data validation automation frameworks using Python with seamless CI/CD pipeline integration.
Key Responsibilities:
- Perform thorough data pipeline testing across complex data systems and enterprise-grade data platforms.
- Develop and execute advanced SQL queries to validate, reconcile, and transform data.
- Design and implement data validation automation frameworks using Python, integrating them seamlessly into CI/CD pipelines.
- Collaborate with DevOps teams to automate workflows and contribute to version-controlled repositories on GitHub.
- Participate in performance testing and monitoring, ensuring data integrity across structured and semi-structured datasets.
Requirements:
- Mandatory experience in ETL testing and advanced SQL for large-scale data operations.
- Proven ability to automate data validation using Python, with exposure to CI/CD tools and GitHub.
Bonus Skills:
- Working knowledge of AWS cloud services, especially S3 and Airflow.
- Experience with PostgreSQL, Oracle, or SQL Server databases.
-
Data Pipeline Architect
2 weeks ago
Davangere, Karnataka, India beBeeDataSystems Full time ₹ 15,00,000 - ₹ 25,00,000Job Title:Data Systems EngineerWe are seeking skilled professionals to design, develop, and maintain robust data pipelines that power cloud-native solutions.Main Responsibilities:Design, build, and manage ETL/ELT workflows in cloud environments.Leverage AWS services for scalable data ingestion, transformation, storage, and querying.Ensure robust data...
-
Data Architect for Cloud Data Pipelines
2 weeks ago
Davangere, Karnataka, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 17,00,000Cloud Data Engineer Job DescriptionA Data Architect for Cloud Data Pipelines is needed to design, build, and maintain robust data pipelines for processing channel activity data.This role involves working on AWS cloud infrastructure and orchestration tools like Airflow for high availability, scalability, and performance.Main Responsibilities:Designing,...
-
Designing Robust Data Pipelines
1 week ago
Davangere, Karnataka, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 20,00,000Senior Data Engineering RoleCrafted solutions drive business forward by harnessing the power of data. As a Senior Data Engineer, your expertise will be pivotal in designing and refining intricate data pipelines to inform strategic decisions.Create and enhance ETL workflows for efficient processing of large datasets.Maintain data accuracy, consistency, and...
-
Chief Data Pipeline Architect
2 weeks ago
Davangere, Karnataka, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000Job OverviewA Senior Data Engineer is required to design, build and optimize scalable data pipelines on AWS.Design and build scalable data pipelines using cloud technologies such as AWS and Apache Spark for big data processing.Implement data ingestion, transformation and integration solutions using cloud services like S3, Glue and SQL.Manage and optimize...
-
Distinguished Data Pipeline Specialist
2 weeks ago
Davangere, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000We are seeking an experienced Data Engineer to contribute to our team's product engineering efforts.Key Responsibilities:Create and maintain efficient, reusable, and scalable data pipelines using AWS services.Write Python code to automate tasks and enhance data processing efficiency.Collaborate with SQL databases for data modeling, queries, and performance...
-
Senior GCP Data Pipeline Specialist
2 weeks ago
Davangere, Karnataka, India beBeeDataEngineer Full time ₹ 8,00,000 - ₹ 15,00,000Job Title:Sr GCP Data EngineerDesign, develop and maintain data pipelines on Google Cloud Platform.Implement scalable data storage solutions and optimize complex data processing workflows.Evaluate and ensure high-quality data throughout the entire data lifecycle.Collaborate with cross-functional teams including data scientists and analysts to understand data...
-
Senior Data Pipeline Specialist
2 weeks ago
Davangere, Karnataka, India beBeePipeline Full time ₹ 12,00,000 - ₹ 17,00,000Job Overview:The role of a Data Pipeline Architect involves designing and implementing scalable data solutions using Python and GCP services. This full-time position requires an experienced professional to lead the development and deployment of reliable datasets and insights.Key Responsibilities:Data Solution Design: Develop scalable data solutions using...
-
Senior Data Engineer
1 week ago
Davangere, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000Data Engineer Position OverviewWe are seeking an experienced Data Engineer to join our team. As a key member of the data engineering group, you will be responsible for developing and maintaining large-scale data pipelines using cloud-based technologies.Key Responsibilities:• Develop and implement data pipelines using Google Cloud technologies such as...
-
Davangere, Karnataka, India beBeeCloudDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000Job Opportunity:We are seeking a highly skilled Cloud Data Engineer with 5+ years of experience in designing, building, and optimizing scalable data architectures on Google Cloud Platform (GCP).Design and implement efficient data pipelines using Google Cloud Storage (GCS), BigQuery, and Airflow.Develop robust ETL/ELT processes using Python and integrate with...
-
Elasticsearch Data Pipeline Architect
2 weeks ago
Davangere, Karnataka, India beBeeSolution Full time ₹ 1,20,00,000 - ₹ 2,50,00,000Job Title: SRE Elasticsearch Administrator">This role involves continually evaluating the performance of our Elasticsearch and Splunk clusters to identify potential issues, planning changes for high-load events, applying security fixes, testing incremental upgrades, and enhancing monitoring and alert infrastructure.Evaluate cluster performance, anticipate...