Professional Data Pipeline Specialist

21 hours ago

Aurangabad, Maharashtra, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000
Job Title: Data Engineer

We are seeking a highly skilled professional to lead our data engineering efforts. As a Data Engineer, you will be responsible for designing, developing, and deploying scalable data pipelines and ETL processes.

  • Lead the design, development, and deployment of scalable data pipelines and ETL processes using Scala and Apache Spark.
  • Develop and maintain data processing workflows using PySpark and Python.
  • Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver robust data solutions.
  • Architect and implement data ingestion, transformation, and integration solutions for structured and unstructured data.
  • Optimize Spark jobs for performance and cost efficiency on cloud platforms.
  • Lead, mentor, and guide a team of data engineers, ensuring best practices in coding, testing, and deployment.
  • Establish and enforce data engineering standards, code reviews, and documentation.
  • Troubleshoot and resolve complex data pipeline issues.
  • Stay current with emerging technologies and recommend improvements to existing data infrastructure.
  • Work closely with DevOps and cloud teams to ensure smooth deployment and monitoring of data pipelines.

Required Qualifications:

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
  • 10+ years of professional experience in data engineering.
  • Strong hands-on programming experience in Scala, with a deep understanding of functional programming concepts.
  • Extensive experience with Apache Spark, including Spark SQL, Spark Streaming, and optimization techniques.
  • Proficiency in Python and PySpark for data processing tasks.
  • Experience working with Databricks or similar cloud-based Spark platforms.
  • Solid understanding of distributed computing, big data architectures, and data modeling.
  • Experience with cloud platforms such as AWS, Azure, or GCP.
  • Familiarity with data storage technologies like HDFS, S3, Delta Lake, and relational/non-relational databases.
  • Strong leadership skills with experience leading and mentoring engineering teams.
  • Excellent problem-solving, communication, and collaboration skills.

  • High-Performance Data Pipeline Specialist

    6 days ago

    Aurangabad, Maharashtra, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Data EngineerWe are seeking a skilled Data Engineering Specialist with experience in designing and implementing scalable, high-performance data pipelines using cloud services like AWS and Databricks.Key Responsibilities:Design, develop, and maintain efficient data workflows using AWS and Databricks.Build and optimize ETL/ELT processes for large-scale...

  • Big Data Pipeline Specialist

    4 days ago

    Aurangabad, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data EngineerJob Summary:We are seeking a skilled data engineer to fill this role at our organization. The ideal candidate will have 1–2 years of experience in data engineering and be proficient in building production-grade pipelines at scale.About the Role:Design, build, and maintain production-level data pipelines that handle terabytes of...

  • End-To-End Data Pipeline Developer

    7 days ago

    Aurangabad, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    At Xebia, we are seeking a skilled professional to join our team. The successful candidate will have expertise in building end-to-end data pipelines and delivering scalable data solutions on Azure.Key Responsibilities:Design, develop, and run complete E2E Data PipelinesHandle data validation, transformation, and warehousingCollaborate with stakeholders for...

  • Data Flow Specialist

    2 days ago

    Aurangabad, Maharashtra, India beBeeDataFlow Full time ₹ 60,00,000 - ₹ 1,00,00,000

    Job Title: Data Flow SpecialistKey Responsibilities:Design and develop data pipelines using Oracle Data Flow or Apache Spark (PySpark) with solid understanding of data patterns, Python, and SQL.Orchestrate data workflows ensuring seamless integration and automation across data services.Utilize cloud infrastructure components to manage data lakes, warehouses,...

  • Lead Data Specialist

    6 days ago

    Aurangabad, Maharashtra, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Lead Data SpecialistWe are seeking an experienced Lead Data Specialist to design, build and optimize data pipelines using Azure Cloud technologies.

  • Data Engineering Specialist

    1 week ago

    Aurangabad, Maharashtra, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 24,00,000

    Data Engineering SpecialistWe are seeking a skilled data specialist to develop and maintain ETL pipelines, implement client-specific data transformations, and optimize database queries.Key Responsibilities:Develop and Maintain ETL PipelinesImplement Client-Specific Data TransformationsSupport Data Ingestion and Quality ChecksOptimize Database...

  • Senior Data Pipeline Developer

    6 days ago

    Aurangabad, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Senior Data Engineer – Data PipelinesAbout the RoleWe are seeking a highly skilled and experienced Senior Data Engineer to join our data engineering team. You will be responsible for designing, developing, and maintaining scalable and efficient data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.Key...

  • Transformative Data Science Specialist

    7 days ago

    Aurangabad, Maharashtra, India beBeeMachine Full time ₹ 22,26,000 - ₹ 29,97,900

    Unlock the Potential of Artificial Intelligence as a Data Science Specialist.About the Role:As a Data Science Specialist, you will play a key part in harnessing the power of artificial intelligence to drive business outcomes. You will work closely with our team to design and develop AI-powered solutions that transform data into actionable insights.Key...

  • AI Driven Data Pipelines Architect

    2 weeks ago

    Aurangabad, Maharashtra, India beBeeData Full time ₹ 10,00,000 - ₹ 15,00,000

    About Our PlatformWe are building a cutting-edge, AI-driven trading platform that ingests vast streams of financial, alternative, and sentiment data and transforms them into actionable insights for informed decision-making.Our mission is to integrate robust data engineering pipelines with Generative AI and advanced ML models to democratize access to...

  • Highly Skilled Data Solutions Professional

    7 days ago

    Aurangabad, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Scalable Data Solutions SpecialistWe are seeking a highly skilled Scalable Data Solutions Specialist to build and optimize scalable ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake.This role involves working across scrum teams to develop data solutions, ensure data governance with Unity Catalog, and support real-time and batch processing.Key...