Data Pipeline Architect

3 days ago


Ghaziabad, Uttar Pradesh, India beBeeData Full time ₹ 60,00,000 - ₹ 1,20,00,000
Job Title: Data Pipeline Architect

We are seeking an experienced professional to design, develop, and optimize data ingestion pipelines.

About the Role:
  • Design and implement data ingestion pipelines for integrating multiple sources into Databricks.
  • Implement and maintain CI/CD pipelines for data workflows.
  • Deploy and manage containerized applications using Kubernetes.
Key Responsibilities:
  1. Data Ingestion: Design and implement scalable and efficient data ingestion pipelines for integrating multiple sources into Databricks.
  2. CICD Pipelines: Implement and maintain CI/CD pipelines for data workflows ensuring seamless integration with existing infrastructure.
  3. Kubernetes Management: Deploy and manage containerized applications using Kubernetes ensuring high availability and scalability.
  4. Data Integrity: Ensure data integrity throughout the ingestion process by implementing robust quality control measures.
  5. Collaboration: Collaborate with Data Engineers and other stakeholders to streamline data ingestion strategies and troubleshoot pipeline issues.
Required Skills & Qualifications:
  • Expertise in data ingestion and pipeline development.
  • Hands-on experience with CI/CD tools (GitHub Actions, Jenkins, Azure DevOps, etc.).
  • Strong knowledge of Kubernetes and container orchestration.
  • Experience with Databricks, Spark, and data lake architectures.
  • Proficiency in Python, Scala, or SQL for data processing.
  • Familiarity with cloud platforms such as AWS, Azure, or GCP.
Benefits:
  • Opportunity to work on complex data ingestion projects.
  • Collaborative environment with experienced professionals.
  • Professional growth and development opportunities.


  • Ghaziabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 22,00,000

    Azure Data Engineer RoleSeeking a skilled data engineer to design, build and maintain automated data pipelines supporting business intelligence and analytics initiatives.We're looking for an expert who can leverage their knowledge of Azure Databricks, PySpark, SQL and ETL/ELT pipeline design to develop robust data pipelines that ingest, transform and curate...


  • Ghaziabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 20,00,000

    We are looking for a skilled Data Engineer to join our team.About the RoleThis is a challenging opportunity for a seasoned Data Engineer to design, implement and maintain robust data pipelines using CI/CD practices, Databricks (Spark), Python and SQL.The ideal candidate will have hands-on experience with modern DevOps/configuration management tools and...

  • Data Engineer

    1 week ago


    Ghaziabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    ETL Developer RoleWe are seeking an experienced ETL Developer to join our data engineering team.This role involves designing, developing, and maintaining scalable and efficient data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.About the RoleThe ideal candidate will have 4+ years of experience in ETL development with at least...

  • Data Architect

    4 days ago


    Ghaziabad, Uttar Pradesh, India beBeeDataArchitect Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data ArchitectWe are seeking a highly skilled and experienced Data Architect to build and design scalable data systems using Azure Databricks, PySpark, and Delta Lake.Collaborate with cross-functional teams to develop high-quality database solutions for the organization.Create and implement scalable data pipelines with Azure Databricks by...


  • Ghaziabad, Uttar Pradesh, India beBeeData Full time US$ 2,00,000 - US$ 2,50,000

    Senior Data EngineerOur mission is to make nutritious food accessible and affordable for everyone. As a critical member of our team, you will be pivotal in designing, building, and maintaining highly scalable data pipelines, optimizing data delivery, and automating data processes.About the RoleThis role is responsible for constructing and optimizing our data...

  • Data Architect

    4 days ago


    Ghaziabad, Uttar Pradesh, India beBeeData Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Data Architect PositionWe are seeking a skilled professional to design, develop and maintain large-scale data systems using distributed computing frameworks.Key Responsibilities:Evaluate existing system architectures for scalability and efficiency.Implement database performance tuning techniques to improve system speed.Collaborate with cross-functional teams...

  • Data Architect

    6 days ago


    Ghaziabad, Uttar Pradesh, India beBeeDataArchitect Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Data ArchitectWe are seeking an experienced data architect to join our analytics division, aligned with our data analytics and business intelligence vertical.Conceptualize and own the build out of problem-solving data marts for consumption by data science and BI teams, evaluating design and operational tradeoffs within systems.Design, develop, and...


  • Ghaziabad, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Job DescriptionThreatXIntel is a startup cyber security company that focuses on protecting businesses and organizations from cyber threats. Our team offers various services, such as cloud security, web and mobile security testing, and DevSecOps.We are seeking an experienced Freelance Data Engineer with strong expertise in Python and AWS services (S3, Glue,...

  • Data Architect

    5 days ago


    Ghaziabad, Uttar Pradesh, India Veritis Group Inc Full time

    Position Title: Data ArchitectLocation: HyderabadEmployment Type: Full-TimeMode: Work from OfficeWorking Shift: UK but might change based on business needAbout the RoleWe are looking for a hands-on Data Architect who can own the end-to-end data architecture strategy while also actively engage in the design, development, and optimization of our data...


  • Ghaziabad, Uttar Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job OverviewWe are seeking a highly skilled and experienced Data Engineer to drive our data strategy.This role is responsible for designing, developing, and maintaining scalable data pipelines using PySpark.Design and develop complex data pipelines utilizing PySpark and Spark SQL.Collaborate with cross-functional teams to ensure accurate data processing,...