Data Pipeline Specialist

5 days ago


Dombivli, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000
Job Title: Senior Data Engineer

Job Description:

We are seeking an experienced and skilled Data Engineer to join our team and contribute to the design, development, and maintenance of large-scale data pipelines using cloud-based technologies.

The ideal candidate will have a strong understanding of Python, data processing, and cloud infrastructure. They will work closely with stakeholders and business clients to identify opportunities for data acquisition and translate business needs into technical requirements.

Key Responsibilities:

  • Design, develop, test, and maintain data acquisition pipelines for large volumes of structured and unstructured data.
  • Build complex datasets based on business requirements using cloud-based tools.
  • Construct scalable 'big data' pipeline architecture.
  • Identify opportunities for data acquisition via collaboration with stakeholders and business clients.
  • Translate business needs into technical requirements.
  • Use logs & alerts to effectively monitor pipelines.
  • Develop JSON messaging structure for integrating with various applications.
  • Leverage DevOps and CI/CD practices (GitHub, Terraform) to ensure the reliability and scalability of data pipelines.


  • Dombivli, Maharashtra, India beBeeAzure Full time ₹ 12,00,000 - ₹ 20,10,000

    Job DescriptionThreatXIntel, a cybersecurity company dedicated to protecting businesses from cyber threats, is seeking an experienced Freelance Azure Data Factory (ADF) Specialist to support our data integration and ETL needs.The ideal consultant will have strong hands-on expertise in Azure ADF pipelines, data orchestration, and transformations to ensure...


  • Dombivli, Maharashtra, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 24,00,000

    Cloud Data EngineerWe are seeking a skilled and motivated cloud data engineer to join our team.The ideal candidate will be proficient in designing, developing, and optimizing robust data ingestion pipelines using GCP services such as Pub/Sub, Dataflow, and Cloud Storage. They will also have experience in architecting and managing scalable BigQuery data...


  • Dombivli, Maharashtra, India beBeePipeline Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data Pipeline ArchitectOverview:We are seeking a skilled data pipeline architect to design, develop, and manage data pipelines that drive business growth. The ideal candidate will have strong expertise in designing and developing data pipelines using Oracle Data Flow or Apache Spark (PySpark), with a solid understanding of data patterns, Python,...


  • Dombivli, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Overview:A Data Engineer at a top organization designs and develops scalable data pipelines using AWS services like Glue, S3, and Redshift. This role requires a strong understanding of Python, PySpark, and SQL to build robust ETL processes.Key ResponsibilitiesDesign and implement data pipelines using PySpark, focusing on performance, scalability, and...


  • Dombivli, Maharashtra, India beBeeDataEngineer Full time US$ 1,00,000 - US$ 1,50,000

    Freelance Data Engineer OpportunityJoin a collaborative environment where you can utilize your expertise in designing and optimizing large-scale data pipelines.Role OverviewAs an experienced Freelance Data Engineer, you will work closely with internal teams to design, build, and maintain ETL/ELT pipelines using PySpark, AWS, and Databricks. The ideal...


  • Dombivli, Maharashtra, India beBeePython Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Python EngineerWe are seeking a seasoned Senior Python Engineer to lead our engineering group. As a key member of the team, you will be responsible for designing and developing scalable data pipelines using Python and Pandas.You will work closely with Data Scientists, Analysts, and Engineering teams to support data-driven applications and reporting...


  • Dombivli, Maharashtra, India beBeeData Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Opportunity:Seeking a seasoned Data Architect to lead the design and implementation of scalable data pipelines.Key Responsibilities:Develop and maintain robust ETL/ELT pipelines using Spark, PySpark, SQL, and PythonOptimize large-scale data processing workflows for performance and reliabilityCollaborate with cross-functional teams on AWS services such as...


  • Dombivli, Maharashtra, India beBeeDataIntegration Full time ₹ 15,00,000 - ₹ 20,10,000

    Job Title: Data Integration SpecialistOur organization seeks a highly skilled Data Integration Specialist to join its team. The ideal candidate will have expertise in designing, developing, and optimizing data ingestion pipelines for integrating multiple sources into Databricks.Key Responsibilities:Design, develop, and optimize data ingestion pipelines for...


  • Dombivli, Maharashtra, India beBeeDataValidation Full time ₹ 9,00,000 - ₹ 12,00,000

    Job Title: Data Validation SpecialistJob Description:A data validation specialist is required to execute automated testing and validation of large-scale data pipelines. This position involves thorough analysis, technical skillset in QA automation, software development lifecycle understanding, agile settings familiarity, and strong SQL skills.Key...


  • Dombivli, Maharashtra, India beBeeGrowth Full time ₹ 5,00,000 - ₹ 10,00,000

    Job TitleDemand Generation Specialist is a role that drives pipeline growth and campaigns through the execution of end-to-end demand generation plans.