Lead Data Pipeline Developer

2 weeks ago

Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,50,00,000
Job Description

We are seeking a skilled Data Engineer to join our team in Mumbai. The ideal candidate will have strong experience in PySpark and hands-on expertise in building scalable data pipelines.

The successful candidate will work on designing, developing, and maintaining batch and streaming data pipelines using Spark SQL, DataFrame, and RDD APIs. They will also have proficiency in using Apache Airflow, dbt, Delta Lake, and other tools/technologies.

Additionally, the Data Engineer should have experience working with cloud services such as AWS S3 (data lake), AWS Glue / EMR (for Spark workloads), AWS Lambda, Step Functions, or similar for orchestration/integration.

Required Skills and Qualifications
  • Strong Experience in PySpark
  • Hands-on expertise in building scalable data pipelines using PySpark.
  • Proficiency in using Spark SQL, DataFrame, and RDD APIs to implement complex business logic.
  • Ability to choose and apply the right Spark APIs (DataFrame, Dataset, RDD) for performance-efficient implementation of business logic at scale.
Benefits

As a Data Engineer at our company, you will have the opportunity to work on challenging projects and contribute to the growth of our team. You will also receive comprehensive training and support to help you succeed in your role.

Others

Our team is passionate about delivering high-quality solutions that meet the needs of our customers. We prioritize collaboration, innovation, and continuous learning. If you share our values and are looking for a challenging and rewarding role, we encourage you to apply.

  • Data Pipeline Developer

    2 weeks ago

    Palakkad, Kerala, India beBeeIngestion Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OpportunityWe are seeking an Ingestion Specialist with expertise in data pipeline development and Kubernetes to fill a key role in our organization.This position requires a strong understanding of data ingestion processes, focusing on integrating various data sources into Databricks.The ideal candidate will have hands-on experience with data pipelines,...

  • Senior Data Pipeline Developer

    5 days ago

    Palakkad, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Data Engineer PositionWe are seeking a skilled Data Engineer to join our team. As a key member of the data team, you will be responsible for designing and developing scalable data pipelines using Google Cloud Platform (GCP) tools.This is a remote opportunity that requires 10+ years of experience in data engineering with a focus on real-time and big query...

  • Data Engineer for Scalable Data Pipelines

    1 week ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 19,00,000

    We are seeking a highly skilled Data Engineer to lead the design and development of scalable data pipelines. The ideal candidate will have expertise in data engineering principles, with a strong understanding of cloud-based data platforms.Key Responsibilities:Design and develop efficient data pipelines using Airflow and SnowflakeCollaborate with...

  • Sales Pipeline Developer

    4 days ago

    Palakkad, Kerala, India beBeeInsurance Full time ₹ 36,00,000 - ₹ 72,00,000

    Job Role:Pipeline Development SpecialistDevelop and maintain a strong sales pipeline by collaborating with the sales team. Ensure consistent follow-ups and timely closures of leads.Identify potential clients through research and proactive outreach to foster growth.Conduct initial qualification of leads to assess their needs and potential, aligning them with...

  • Data Pipeline Architect

    2 weeks ago

    Palakkad, Kerala, India beBeeETLDeveloper Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    We are seeking a skilled professional to fill the role of an ETL Specialist with expertise in designing and optimizing complex data pipelines using Talend Open Studio.Key qualifications for this position include 8 to 10 years of experience working with large datasets, proficiency in SQL and Python programming languages, as well as hands-on experience with...

  • Expert Data Pipeline Architect

    7 days ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000

    Job Title:A Data Engineer is needed to design, build, and maintain large-scale data pipelines using Google Cloud Platform.Responsibilities:Develop, construct, test, and maintain data acquisition pipelines for structured and unstructured data in GCP.Create complex datasets based on business requirements.Construct 'big data' pipeline architecture.Identify...

  • Chief Data Pipeline Architect

    6 days ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineer Role OverviewSeeking a skilled data engineer to design, develop, and maintain robust data pipelines. Responsibilities include extracting data from Oracle Symphony via APIs, processing and storing it in the Databricks Lakehouse platform, and integrating it into Oracle EPM.Key Responsibilities:Design and implement end-to-end data pipelines from...

  • AWS Data Architect

    5 days ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    We are seeking an experienced AWS Data Engineer to design, develop and maintain robust data pipelines using Apache Spark, PySpark, SQL and Python.Key Responsibilities:Develop scalable ETL/ELT pipelines on AWS using Spark and PySparkOptimize large-scale data processing workflows for performance and reliabilityWork with AWS services such as Glue, EMR, S3,...

  • Data Pipeline Specialist

    1 week ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Data Engineering ProfessionalWe are seeking a skilled data engineer to join our analytics division.You will be part of our data analytics and business intelligence vertical, responsible for conceptualizing and owning the build out of problem-solving data marts for consumption by data science and business intelligence teams. You will evaluate design and...

  • Senior Data Engineer

    2 weeks ago

    Palakkad, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job DescriptionWe are seeking a highly skilled professional to join our team as a Big Data Engineer. In this role, you will be responsible for designing, building, and maintaining robust data pipelines that handle high-volume financial data.The ideal candidate will have experience with data pipeline tools, cloud platforms, and data warehousing. They will...