Leading Data Pipelines Specialist

3 days ago

Ahmedabad, Gujarat, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,10,000
Job Description

This is a challenging opportunity for a skilled Data Engineer to join our team at dxf.

  • Design, develop, and maintain scalable data pipelines for batch and streaming workflows.
  • Implement robust ETL/ELT processes to extract data from various sources and load into data warehouses.
  • Build and optimize database schemas following best practices in normalization and indexing.
  • Create and maintain documentation for data flows, pipelines, and processes.
  • Collaborate with cross-functional teams to translate business requirements into technical solutions.
  • Monitor and troubleshoot data pipelines to ensure optimal performance.
  • Implement data quality checks and validation processes.
  • Build and maintain CI/CD workflows for data engineering projects.
  • Stay current with emerging technologies and recommend improvements to existing systems.

Requirements

  • Bachelor's degree in Computer Science, Information Technology, or related field.
  • Minimum 4+ years of experience in data engineering roles.
  • Strong proficiency in Python programming and SQL query writing.
  • Hands-on experience with relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra).
  • Experience with data warehousing technologies (e.g., Snowflake, Redshift, BigQuery).
  • Proven track record in building efficient and scalable data pipelines.
  • Practical knowledge of batch and streaming data processing approaches.
  • Experience implementing data validation, quality checks, and error handling mechanisms.
  • Working experience with cloud platforms, particularly AWS (S3, EMR, Glue, Lambda, Redshift) and/or Azure (Data Factory, Databricks, HDInsight).
  • Understanding of different data architectures including data lakes, data warehouses, and data mesh.
  • Demonstrated ability to debug complex data flows and optimize underperforming pipelines.
  • Strong documentation skills and ability to communicate technical concepts effectively.

  • Data Pipeline Architect

    7 days ago

    Ahmedabad, Gujarat, India beBeeIngestion Full time ₹ 1,80,00,000 - ₹ 2,40,00,000

    Enterprise Data Ingestion SpecialistThe position entails designing, developing, and optimizing scalable data ingestion workflows to integrate multiple sources into Databricks. Key responsibilities include implementing and maintaining CI/CD pipelines for data workflows, deploying and managing containerized applications using Kubernetes, and ensuring data...

  • Expert Data Pipelines Architect

    7 days ago

    Ahmedabad, Gujarat, India beBeeDataIntegration Full time ₹ 25,00,000 - ₹ 30,00,000

    We are seeking a talented professional to fill the role of a Data Integration Specialist with expertise in designing and optimizing complex data pipelines using Talend Open Studio.**Position Details:**- Role: Data Integration Specialist- Experience: 8 to 10 years- Work Timings: Flexible hours, remote work- Duration: 3 Months (Contract)**Key Skills...

  • Data Pipeline Developer

    4 days ago

    Ahmedabad, Gujarat, India beBeeTechnical Full time ₹ 60,00,000 - ₹ 1,20,00,000

    Job DescriptionWe are seeking a highly skilled technical professional to design, implement, and manage data pipelines using Azure Data Factory.As a technical specialist, you will be responsible for developing scalable data processing solutions within Azure Synapse Analytics, with a strong understanding of data modeling, ETL processes, and query...

  • Senior Data Pipeline Specialist

    20 hours ago

    Ahmedabad, Gujarat, India beBeeData Full time ₹ 8,00,000 - ₹ 12,00,000

    Optimizing data pipelines is key to ensuring seamless data integration. The successful ETL Tester will have strong analytical skills, attention to detail, and excellent problem-solving abilities.The ideal candidate should be proficient in various data transfer tools such as Azure/ADF, SQL, and Python or Java. He/she must be able to identify and troubleshoot...

  • Master Data Pipeline Architect

    3 days ago

    Ahmedabad, Gujarat, India beBeeDataEngineer Full time US$ 1,17,000 - US$ 1,63,000

    Key Data Engineer RoleThe ideal candidate will possess a deep understanding of data infrastructure, data modeling, and data governance. They should be well-versed in cloud computing platforms, ELT/ETL processes, and data warehousing concepts.We are seeking an experienced Data Engineer to join our team. The successful candidate will be responsible for...

  • Senior Data Engineer

    19 hours ago

    Ahmedabad, Gujarat, India beBeeDataEngineer Full time ₹ 14,68,800 - ₹ 19,12,000

    Job OpportunityA senior data engineer with expertise in Airflow, Python, and Snowflake is required to design, develop, and maintain complex data pipelines.Able to leverage distributed systems, containerization, and cloud-based technologies.The ideal candidate will have experience working on scalable data processing solutions using Apache Airflow, implement...

  • Anblicks - Lead Data Engineer - Snowflake Migration

    2 days ago

    Ahmedabad, Gujarat, India ANBLICKS CLOUD DATA ENGG PRIVATE LIMITED Full time

    Job Title : Lead Data Engineer Snowflake MigrationLocation : Hyderabad / AhmedabadExperience : 7+ YearsBe a Trailblazer in Our Data Modernization Center of Excellence (COE)At Anblicks, we are on a mission to modernize legacy data ecosystems and help enterprises unlock the power of the cloud. Our Data Modernization COE is at the forefront of this...

  • Chief Data Pipeline Architect

    4 days ago

    Ahmedabad, Gujarat, India beBeeDataEngineer Full time ₹ 9,00,000 - ₹ 12,00,000

    Job OpportunityWe are seeking an experienced Freelance Data Engineer to design, develop, and optimize data pipelines.About Our WorkData engineering is a crucial aspect of our business operations, and we require a skilled professional to oversee data management, processing, and storage.Main ResponsibilitiesDesign and implement efficient data pipelines using...

  • Data Pipeline Engineer

    1 day ago

    Ahmedabad, Gujarat, India beBeeBackOffice Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Description">Role OverviewAbout the Role:We are looking for a skilled software engineer to build robust data pipelines and transform raw on-chain data into actionable insights.This role is ideal for someone who enjoys low-level protocol work, understands DeFi mechanics, and can implement pricing logic from decentralized exchanges...

  • Leading Software Architect for Scalable Data Pipelines

    5 days ago

    Ahmedabad, Gujarat, India beBeeSoftware Full time ₹ 40,00,000 - ₹ 60,00,000

    Key Responsibilities:Lead a team of engineers in the design and implementation of scalable data pipelines using Apache Beam and Databricks.Develop and maintain a shared component library for data ingestion, parsing, extraction, validation, enrichment, and publishing.Define patterns and templates for Apache Beam pipelines and Databricks jobs, standardizing...