Senior Data Pipeline Specialist

2 weeks ago

Solapur, Maharashtra, India beBeeBigData Full time ₹ 20,00,000 - ₹ 25,00,000
Job Opportunity

We are seeking an experienced Big Data Solutions Architect to join our team. As a key member of our data engineering team, you will design, develop, and optimize large-scale data pipelines and distributed data processing systems.

The ideal candidate will have a strong background in Scala, Spark (PySpark), and Python, as well as hands-on experience with Apache Kafka, NiFi / Airflow, and cloud-based solutions.

Key Responsibilities
  • Design and develop scalable big data architectures using Scala, Spark, and Python.
  • Work closely with cross-functional teams to integrate streaming solutions using Apache Kafka and orchestration tools like NiFi / Airflow.
  • Write efficient queries and perform data analysis using Jupyter Notebooks and SQL.
  • Collaborate with the team to ensure delivery of high-quality code through code reviews, performance tuning, and best practices.
  • Build monitoring and alerting systems leveraging Splunk or equivalent tools.
Mandatory Skills
  • Scala – Minimum 3+ years of hands-on experience.
  • Strong expertise in Spark (PySpark) and Python.
  • Hands-on experience with Apache Kafka.
  • Knowledge of NiFi / Airflow for orchestration.
  • Strong experience in Distributed Data Systems (5+ years).
  • Proficiency in SQL and query optimization.
Preferred Skills
  • Exposure to messaging technologies like Apache Kafka or equivalent.
  • Experience in designing intuitive UIs for data analytics visualization.
  • Familiarity with Splunk or other monitoring/alerting solutions.

  • Senior Data Pipeline Specialist

    2 weeks ago

    Solapur, Maharashtra, India beBeedataengineer Full time ₹ 15,00,000 - ₹ 25,00,000

    About the RoleWe are seeking a skilled Data Pipeline Engineer to build scalable and efficient data pipelines.You will work on creating robust data systems using IBM DataStage, AWS Glue, and Snowflake.

  • GCP Data Pipeline Specialist

    2 weeks ago

    Solapur, Maharashtra, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior GCP Data Engineer PositionWe're looking for a seasoned data professional with extensive experience in designing, developing, and maintaining scalable data pipelines on the Google Cloud Platform. The ideal candidate will have a deep understanding of GCP services, including Big Query, Cloud Dataflow (Apache Beam), Cloud Run, Cloud Functions, Pub/Sub,...

  • Data Pipeline Architect

    2 weeks ago

    Solapur, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Data Insights SpecialistAbout the RoleWe are seeking a highly skilled Data Insights Specialist to join our team. The successful candidate will be responsible for developing and maintaining complex data pipelines, designing and implementing dataflows, and ensuring efficient data processing and analysis.The ideal candidate will have proficiency in...

  • Data Pipeline Specialist

    2 weeks ago

    Solapur, Maharashtra, India beBeeDataEngineering Full time ₹ 25,00,000 - ₹ 35,00,000

    Data Engineer RoleAs a Data Engineer, you will be responsible for designing, building, and maintaining robust data pipelines that power our Agentic AI solutions on AWS.The role focuses on developing efficient ETL/ELT workflows, ensuring data quality, security, and scalability to support AI/ML model training, inference, and intelligent decision-making...

  • Senior Data Pipeline Developer

    2 weeks ago

    Solapur, Maharashtra, India beBeeDataPipelineDeveloper Full time ₹ 15,00,000 - ₹ 25,00,000

    We are seeking a highly skilled Data Pipeline Developer to design, develop and maintain scalable ETL pipelines using Azure services.The ideal candidate will have extensive experience with cloud-based analytics, data engineering and a strong background in data warehousing. They will be responsible for designing, developing and maintaining efficient data...

  • Building Scalable Data Pipelines

    2 weeks ago

    Solapur, Maharashtra, India beBeeDataMigration Full time ₹ 8,00,000 - ₹ 15,00,000

    Job Title: Data Migration SpecialistAbout the ProjectThis role is critical for ensuring a smooth transition of a Fortune 500 client's large-scale database migration initiative. The project involves transitioning from Fivetran to Nexla, with Snowflake as the target data platform.The successful candidate will have hands-on experience with data migration and...

  • Principal Data Pipeline Specialist

    2 weeks ago

    Solapur, Maharashtra, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 20,10,000

    Data Engineer Position OverviewWe are seeking a highly skilled and experienced Data Engineer to join our team. In this role, you will have the opportunity to work on complex data engineering projects using Databricks.Key Responsibilities:Design, develop, and deploy scalable data pipelines and ETL processes in Databricks.Collaborate with stakeholders to...

  • Senior Data Validator Position

    2 weeks ago

    Solapur, Maharashtra, India beBeeDataValidation Full time ₹ 18,00,000 - ₹ 20,00,000

    Senior Data Validation SpecialistWe are seeking an experienced Senior Data Validation Specialist to join our team. As a key member of the data team, you will be responsible for ensuring the accuracy and integrity of our data pipelines.Key Responsibilities:Data Profiling: Analyze source data to understand its structure, distribution, and anomalies, and ensure...

  • Senior Data Engineer

    2 weeks ago

    Solapur, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    We're looking for a seasoned data engineer to create and maintain large-scale, high-performance data pipelines on the AWS platform.Key Responsibilities:Develop and optimize robust ETL/ELT processes using Spark, PySpark, SQL, and PythonCollaborate with AWS services like Glue, EMR, S3, Lambda, and Redshift to build scalable data architecturesEnsure data...

  • Data Pipeline Architect

    2 weeks ago

    Solapur, Maharashtra, India beBeeAzure Full time ₹ 10,00,000 - ₹ 15,00,000

    Integration Solutions ExpertThe ideal candidate will possess expertise in designing and developing data pipelines using Azure Data Factory (ADF) for streamlined data transfer.Key responsibilities include:Designing, implementing, and optimizing data pipelines to meet business requirements;Troubleshooting and resolving issues related to Azure Synapse...