Senior Data Pipeline Architect

5 days ago

Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

Unlock the Power of Data Engineering

We're seeking an accomplished Data Engineer to champion data-driven initiatives and deliver high-quality solutions.

This pivotal role involves designing, building, and optimizing scalable pipelines using Spark, PySpark, and SQL, collaborating with data scientists and business stakeholders to drive business value.

  • Design and develop efficient ETL workflows and data pipelines utilizing Spark, PySpark, and SQL.
  • Process large datasets leveraging Spark and SQL, ensuring data quality and performance.
  • Build and maintain real-time streaming applications using Spark Streaming/Kafka, driving insights from complex data sets.
  • Collaborate with cross-functional teams to integrate AI/ML models into production, enhancing business outcomes.
  • Develop and implement robust data transformations, streamlining data processing and analysis.

The ideal candidate will possess in-depth knowledge of Python, Spark, and PySpark, backed by hands-on experience in ETL pipeline development, real-time streaming, and data transformation. Additionally, familiarity with performance tuning (partitioning, caching, broadcast joins) and deployment on cloud platforms (AWS, Azure, GCP, Databricks) is essential.

Key Requirements:

Technical Skills: Python, Spark, PySpark, ETL, Real-Time Streaming, Data Transformation

Deployment Platforms: AWS, Azure, GCP, Databricks

Tools and Technologies: Kafka, Flume, Kinesis, Parquet, Avro, ORC

Soft Skills: Collaboration, Communication, Problem-Solving, Adaptability

  • Scalable Data Pipeline Expert

    3 days ago

    Gulbarga, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Data Engineer RoleWe are seeking a seasoned Senior Data Engineer to spearhead the design, development, and maintenance of scalable data pipelines using PySpark. This is an exceptional opportunity for someone with strong expertise in PySpark and hands-on experience in building robust data pipelines.

  • Lead Data Pipeline Architect

    3 days ago

    Gulbarga, Karnataka, India beBeeData Full time ₹ 1,20,00,000 - ₹ 1,60,00,000

    About the RoleWe are seeking an experienced Data Engineer to lead our data pipeline architecture and collaborate with data scientists and analysts.

  • Data Architect

    2 days ago

    Gulbarga, Karnataka, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data Architect">The Data Architect will design, build and maintain large-scale data systems to support real-time and batch data processing. This includes designing scalable data pipelines and infrastructure.">Key Responsibilities:Data Pipeline Design and DevelopmentCloud Infrastructure ManagementBig Data ProcessingDatabase...

  • Chief Data Pipeline Architect

    5 days ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    We are seeking a candidate who is comfortable working on shifts that align with our global operations.Data Engineering OpportunitiesDevelop and maintain data pipelines using advanced frameworks to ensure seamless data flow.Critical Data TransformationsImplement high-touch data transformations for integration to meet client needs.Data Ingestion and...

  • Data Engineer

    2 days ago

    Gulbarga, Karnataka, India beBeeDatastage Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: ETL Lead with IBM DataStageAbout the RoleWe are seeking a motivated and talented ETL developer to join our data engineering team. As an ETL lead, you will work on building scalable and efficient data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.Key Responsibilities:Design, develop, and maintain ETL pipelines using...

  • Senior Data Pipelining Expert

    4 days ago

    Gulbarga, Karnataka, India beBeeData Full time ₹ 60,00,000 - ₹ 1,00,00,000

    About this RoleWe're seeking a Data Transformation Specialist to build robust pipelines that transform large, messy data into high-quality structured data.

  • Data Architect

    5 days ago

    Gulbarga, Karnataka, India beBeeDataArchitecture Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: Data ArchitectWe are seeking a skilled Data Architect to design and implement large-scale data systems.Job Description:The successful candidate will be responsible for developing and maintaining complex data solutions. Experience with Databricks, SQL, PySpark, and Python is essential for success in this role.Our ideal candidate will have hands-on...

  • Senior Data Pipeline Specialist

    3 days ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 80,00,000 - ₹ 1,60,00,000

    Key Roles and Responsibilities:Data Engineers are required to design, develop, and maintain ETL/ELT data pipelines. They must be able to ingest data from various sources into our data warehouse.This includes databases and flat files. The ideal candidate will have experience with AWS Cloud services, including Lambda, DMS, and Glue.A solid understanding of...

  • Cloud Data Solutions Architect

    4 days ago

    Gulbarga, Karnataka, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: Cloud Data Solutions ArchitectWe are seeking a seasoned Cloud Data Solutions Architect to design, develop and maintain scalable, secure and high-performance data solutions on Databricks. Key ResponsibilitiesDesign and implement data pipelines and ETL processes on Databricks.Lead the architecture of cloud data warehouses and lakehouse...

  • Data Architect

    1 week ago

    Gulbarga, Karnataka, India beBeeCloud Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Cloud Data Solutions ArchitectWe are seeking a skilled professional with expertise in Azure Cloud technologies and proficiency in designing scalable data pipelines for batch data ingestion and egress.Key ResponsibilitiesDesign, develop, and deploy ETL workflows utilizing Azure Data Factory and Databricks for high-performance data processing.Implement data...