Current jobs related to Senior Data Pipeline Specialist - Vadodara, Gujarat - beBeeDataEngineer

  • Data Pipeline Quality Assurance Specialist

    2 weeks ago

    Vadodara, Gujarat, India beBeeQuality Full time ₹ 12,00,000 - ₹ 18,00,000

    ETL Specialist: Data Pipeline Quality AssuranceWe are seeking a skilled ETL specialist to ensure the integrity and quality of our data pipelines.Key Responsibilities:With extensive experience in QA with automation, you will identify and report defects in our ETL processes.In an agile setting, you will work closely with cross-functional teams to guarantee...

  • Senior Data Pipeline Specialist

    1 week ago

    Vadodara, Gujarat, India beBeeDataScientist Full time ₹ 18,00,000 - ₹ 25,00,000

    ETL Data ArchitectWe are seeking a seasoned ETL Developer to lead the design and implementation of scalable data pipelines using IBM DataStage, AWS Glue, and Snowflake.Design, develop, and maintain efficient data flows leveraging best practices for schema design, partitioning, and transformation logic.Collaborate with cross-functional teams to ensure timely...

  • Advanced Data Pipeline Architect

    2 weeks ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior ETL Developer Job DescriptionWe are seeking an experienced professional to fill the role of Senior ETL Developer. The successful candidate will have a strong background in data engineering and 6+ years of experience in development and maintenance.Key Responsibilities:Collaborate with stakeholders to understand data requirements and translate them into...

  • Chief Data Pipeline Architect

    1 week ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,10,000

    Key Engineering RoleA highly skilled data engineer is sought to design and build large-scale data pipelines using Google Cloud Platform.Develop, construct, test, and maintain data acquisition pipelines for structured and unstructured data using tools like Python, Data Flow, DataStream, CDC, and BigQuery.Build complex datasets based on business requirements...

  • Cloud Data Pipeline Specialist

    2 weeks ago

    Vadodara, Gujarat, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 20,00,000

    Data Engineer OpportunityUnlock business potential by designing and developing scalable data pipelines using cloud technologies.Main Responsibilities:Design and develop data pipelines using Glue, Python, EMR/Spark, and Kafka/Kinesis streaming.Develop relational databases with a preference for Aurora Postgres.Write complex SQL queries and stored...

  • Senior Data Pipeline Specialist

    2 weeks ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 1,80,00,000

    Data Engineer RoleCodeVyasa is seeking a seasoned Data Engineer to join our team of rockstar developers. This role requires a minimum of 4+ years of experience in designing, developing, and maintaining data pipelines and ETL processes using AWS services.The ideal candidate will have hands-on experience with AWS cloud services (S3, Lambda, EC2, RDS, Glue,...

  • Data Pipeline Architect

    1 week ago

    Vadodara, Gujarat, India beBeePipeline Full time ₹ 8,00,000 - ₹ 13,00,000

    Job Title: Data Pipeline ArchitectAre you an experienced data professional looking to take your career to the next level? We are seeking a skilled Data Pipeline Architect to join our team.Key Responsibilities:Design, develop, and optimize complex data pipelines for integrating multiple sources into Databricks.Implement and maintain continuous...

  • Pipeline Stress Engineer Specialist

    2 weeks ago

    Vadodara, Gujarat, India beBeePipeline Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Pipeline Stress Engineer SpecialistJob Description:As a highly skilled Pipeline Stress Engineer, you will be responsible for designing and analyzing pipeline systems to ensure safe and efficient operation. You will work closely with cross-functional teams to identify potential issues and develop solutions to mitigate risks.Required Skills and...

  • Senior Data Pipeline Architect

    2 weeks ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    About this RoleWe're seeking a seasoned ETL Test Engineer to lead the development of complex data pipelines. This is an exceptional opportunity to collaborate with cutting-edge technologies and contribute to the growth of our team.

  • Senior Data Integration Specialist

    2 weeks ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Summary:This senior data integration specialist position is ideal for someone who enjoys working with large datasets, complex transformations, and efficient loading processes.Key Responsibilities:Data Integration and Transformation: Collaborate with data architects to design and develop ETL solutions using SSIS packages, ensuring seamless data flow and...

Senior Data Pipeline Specialist

2 weeks ago

Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000
Big Data Engineer

We are seeking a highly skilled Big Data Engineer to join our team and support data-driven initiatives. This role involves building scalable pipelines, working with streaming data, and collaborating with data scientists and business stakeholders to deliver high-quality solutions.

  • Key Responsibilities:
  • Design, build, and optimize data pipelines and ETL workflows.
  • Manage and process large datasets using Spark, PySpark, and SQL.
  • Build and maintain real-time streaming applications with Spark Streaming/Kafka.
  • Collaborate with data scientists and product teams to integrate AI/ML models into production.
  • Ensure data quality, scalability, and performance in all pipelines.
  • Deploy and manage Spark workloads on cloud platforms (AWS, Azure, GCP, Databricks).
  • Automate testing and deployment of Spark jobs via CI/CD pipelines.
Required Skills and Qualifications
  • Proven experience in Python, Spark, and PySpark (non-negotiable).
  • Hands-on experience with Spark SQL, DataFrames, UDFs, DAG execution.
  • Knowledge of data ingestion tools (Kafka, Flume, Kinesis) and data formats (Parquet, Avro, ORC).
  • Proficiency in Airflow/Prefect for scheduling and orchestration.
  • Familiarity with performance tuning in Spark (partitioning, caching, broadcast joins).
  • Experience deploying on Databricks, AWS EMR, Azure HDInsight, or GCP Dataproc.
  • Exposure to testing and CI/CD for data pipelines (pytest, Jenkins, GitHub Actions).
Benefits
  • Opportunity to work with a talented team on challenging projects.
  • Professional growth and development opportunities.
  • A competitive salary and benefits package.