Data Pipeline Specialist

6 days ago

Bellary, Karnataka, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000
Data Engineering Role Overview

We are seeking a skilled Data Engineer to join our organization.

As a Data Engineer, you will be responsible for designing, developing, and maintaining efficient data pipelines to ingest data from various sources into a data warehouse.

  • Design and develop optimized data pipelines to extract data from databases and flat files.
  • Utilize AWS services such as AWS Lambda, AWS DMS, and AWS Glue for data ingestion and transformation processes.
  • Develop and optimize Snowflake stored procedures to implement complex business logic and data transformations.
  • Manage and monitor scheduled data workflows using Apache Airflow.

You will work closely with stakeholders to understand project requirements and translate them into technical solutions, ensuring seamless integration with existing data pipelines.

  • Implement and maintain robust data quality checks to ensure accuracy and reliability of all delivered datasets and reports.
  • Adhere to the full project delivery lifecycle, including requirements gathering, design, implementation, documentation, and obtaining user acceptance testing (UAT) approvals.
  • Deliver data sets to other AWS accounts and support the creation of Power BI reports.
Required Skills and Qualifications
  • Proven experience as a Data Engineer or in a similar role.
  • Strong proficiency in SQL scripting.
  • Proficiency in Python for data manipulation and automation.
  • Experience with Apache Airflow for workflow orchestration.
  • Hands-on experience with AWS Cloud services, including Lambda, DMS, and Glue.
  • Solid understanding of data warehousing concepts, with practical experience using Snowflake.
  • Excellent problem-solving skills and the ability to design and implement solutions based on business requirements.
  • Experience with the full software development lifecycle (SDLC) and project documentation.
  • Familiarity with data quality best practices.
  • Strong communication and collaboration skills.
Benefits
  • This role offers an opportunity to work on cutting-edge technology and contribute to the growth of our organization.
Others
  • Please note that we are looking for candidates who can hit the ground running and contribute to our team's success.

  • Data Pipeline Specialist

    4 days ago

    Bellary, Karnataka, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 22,00,000

    Job SummaryWe are seeking a talented Data Engineer to join our team. As a Data Engineer, you will design, build, and maintain production-level data pipelines that handle large amounts of structured and unstructured data.Design and implement scalable data pipelines using AWS services such as S3, EC2, RDS, DynamoDB, Athena, and Redshift.Collaborate with...

  • Data Pipelines Specialist

    2 weeks ago

    Bellary, Karnataka, India beBeeTechnicalConsultant Full time ₹ 80,00,000 - ₹ 1,20,00,000

    Job OverviewThe role involves configuring, integrating, and optimizing financial data workflows within a platform. Collaboration with cross-functional teams is essential for designing reusable and scalable solutions, implementing automation, and enhancing system performance across key investment operations.This position demands hands-on expertise in tools...

  • Senior Data Pipeline Developer

    5 days ago

    Bellary, Karnataka, India beBeeetl Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Job Title: ETL Specialist - Data Pipeline DeveloperExperience: 5–7 YearsLocation: RemoteJob Type: Full-timeAbout the RoleWe are seeking an experienced ETL specialist to join our data engineering team. The ideal candidate will have a strong background in designing, developing, and maintaining large-scale data pipelines using IBM DataStage, AWS Glue, and...

  • Data Pipeline Architect

    1 week ago

    Bellary, Karnataka, India beBeeDataPipeline Full time ₹ 8,00,000 - ₹ 15,00,000

    Job Title: Data Pipeline ArchitectKey Responsibilities:We're seeking an experienced Data Pipeline Architect to design, develop and optimize data ingestion pipelines for integrating multiple sources into Databricks.The successful candidate will implement and maintain CI/CD pipelines for data workflows.Deploy and manage containerized applications using...

  • Scalable Data Pipeline Specialist

    2 weeks ago

    Bellary, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    As a Data Engineer, you will be responsible for designing and developing scalable data pipelines that fuel real-time insights.We are seeking an experienced professional with a strong understanding of data lake and warehouse design principles to join our team.You will be responsible for creating and maintaining cloud-based data lakes, warehouses, and ETL/ELT...

  • Data Pipeline Tester

    6 days ago

    Bellary, Karnataka, India beBeeSenior Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Summary:We are seeking a skilled Senior Test Engineer to join our team. The ideal candidate will have experience in ETL and Power BI testing, with a strong background in Azure Data Factory, Python, and Pytest.The successful candidate will be responsible for designing and developing comprehensive test cases to validate data flows, transformations, and...

  • Data Ingestion Specialist

    3 days ago

    Bellary, Karnataka, India beBeeDataIngestion Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Data Ingestion SpecialistWe are seeking a skilled Data Ingestion Specialist to design, develop, and optimize data ingestion pipelines for integrating multiple sources into Databricks.This role requires expertise in CI/CD and Kubernetes, with a strong understanding of data ingestion processes and integration with cloud platforms.The ideal candidate...

  • Expert Large-Scale Data Pipeline Specialist

    5 days ago

    Bellary, Karnataka, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 25,00,000

    Data Engineering OpportunityWe are seeking a seasoned Freelance Data Engineer with expertise in designing, building, and optimizing large-scale data pipelines.This role involves developing and maintaining ETL/ELT pipelines using PySpark, AWS, and Databricks. Additionally, the successful candidate will perform data profiling, validation, and transformation...

  • Building Scalable Data Pipelines

    2 days ago

    Bellary, Karnataka, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 20,00,000

    Cloud Data EngineerWe are seeking a highly skilled Cloud Data Engineer to join our team. In this role, you will design and build scalable data pipelines using Google Cloud Platform (GCP) services.The ideal candidate will have experience with GCP data processing technologies such as Cloud Functions, Cloud Run, Pub/Sub, BigQuery, and Cloud Storage. You will...

  • Data Flow Specialist

    1 day ago

    Bellary, Karnataka, India beBeeDataFlow Full time ₹ 9,00,000 - ₹ 18,00,000

    Key Role:Data Flow SpecialistAs a seasoned data flow specialist, you will be responsible for designing and developing data pipelines that efficiently integrate and automate data services.Main Responsibilities:Create optimized data pipelines utilizing Oracle Data Flow or Apache Spark (PySpark) to drive business outcomes.Develop strong partnerships with...