Senior Big Data Specialist

3 days ago

Mysore, Karnataka, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

Job Title: Data Architect

This role involves building scalable architectures, working with streaming data, and collaborating with data scientists and business stakeholders to deliver high-quality solutions.

Key Responsibilities:
  • Design, build, and optimize data pipelines and ETL workflows.
  • Manage and process large datasets using Spark, PySpark, and SQL.
  • Build and maintain real-time streaming applications with Spark Streaming/Kafka.
  • Collaborate with data scientists and product teams to integrate AI/ML models into production.
  • Ensure data quality, scalability, and performance in all pipelines.
  • Deploy and manage Spark workloads on cloud platforms (AWS, Azure, GCP, Databricks).
  • Automate testing and deployment of Spark jobs via CI/CD pipelines.
Requirements:
  • Bachelor's/Master's degree in Computer Science, Computer Engineering, or related field.
  • Minimum 4 years of professional experience as a Data Engineer.
  • Strong expertise in Python, Spark, PySpark.
  • Hands-on experience with Spark SQL, DataFrames, UDFs, DAG execution.
  • Knowledge of data ingestion tools (Kafka, Flume, Kinesis) and data formats (Parquet, Avro, ORC).
  • Proficiency in Airflow/Prefect for scheduling and orchestration.
  • Familiarity with performance tuning in Spark (partitioning, caching, broadcast joins).
  • Experience deploying on Databricks, AWS EMR, Azure HDInsight, or GCP Dataproc.
  • Exposure to testing and CI/CD for data pipelines (pytest, Jenkins, GitHub Actions).

  • Big Data Specialist

    3 days ago

    Mysore, Karnataka, India beBeeBigData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Big Data Engineer">Job DescriptionWe are seeking a skilled Big Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining large-scale data processing systems.">Required Skills and QualificationsHands on experience in Python and PySparkExperience in building pySpark...

  • Cloud Native Big Data Architecture Lead

    3 days ago

    Mysore, Karnataka, India beBeeEngineering Full time ₹ 20,00,000 - ₹ 30,00,000

    Senior Big Data Engineer LeaderWe are seeking a highly experienced and strategic leader to head our Big Data platform team. The ideal candidate will have deep expertise in cloud-native environments, Azure Cloud Services, and Databricks.Key Responsibilities:Lead and mentor a team of data engineers working on large-scale distributed data systems.Architect and...

  • Data Architecture Specialist

    11 hours ago

    Mysore, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Architecture SpecialistWe are seeking a skilled Data Architecture Specialist to design, develop and maintain large-scale data processing systems.Key Responsibilities:Design and implement robust data pipelines using PySpark and HiveCollaborate with cross-functional teams to achieve business objectivesImplement data quality tools and ensure...

  • Distinguished Data Architect

    1 day ago

    Mysore, Karnataka, India beBeeData Full time ₹ 80,00,000 - ₹ 1,50,00,000

    Job Title: Senior Software Engineer/Technical SpecialistThis is a highly technical role involving the design, development, and optimization of ETL/ELT pipelines using Azure Data Factory (ADF), Databricks, and Synapse Analytics.Develop and maintain scalable data architectures following Medallion Architecture and best practices.Write and optimize SQL queries...

  • Senior Data Engineering Lead

    24 hours ago

    Mysore, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,10,000

    Job OverviewThis is a senior data engineering position that involves leading a team of big data engineers.The ideal candidate will have excellent problem analysis skills and experience with at least one cloud infra provider such as Azure or AWS.They will also be proficient in building data pipelines using batch processing with Apache Spark or Hive query...

  • Senior Big Data Expert

    1 day ago

    Mysore, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OpportunityWe are seeking a skilled Senior Data Engineer to augment our engineering team. In this role, you will work on the development of a cutting-edge FinTech application designed to scale and meet the needs of users.Key Responsibilities:Bachelor's degree in computer science, information technology, or a related field.4-8 years of hands-on experience...

  • Senior Data Solutions Architect

    3 days ago

    Mysore, Karnataka, India beBeeDataSolutions Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Senior Data Solutions ArchitectAbout the RoleWe are seeking a highly skilled and experienced Senior Data Solutions Architect to join our team. As a key member of our data engineering group, you will be responsible for designing, developing, and maintaining large-scale data pipelines and ETL processes on Databricks.Key ResponsibilitiesDesign and...

  • Senior Data Integration Specialist

    15 hours ago

    Mysore, Karnataka, India beBeeDataIntegration Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Senior Data Integration SpecialistWe are seeking an experienced professional to lead end-to-end data integration projects utilizing Oracle Data Integrator (ODI).Design, develop, and implement ELT processes using ODI.Develop and optimize Knowledge Modules for improved performance.Collaborate with cross-functional teams to integrate ODI with various...

  • Senior Data Infrastructure Specialist

    3 days ago

    Mysore, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Key Responsibilities:We are seeking a skilled Data Engineer who will be responsible for designing and building large-scale data infrastructures. The ideal candidate will have expertise in real-time data processing using technologies like Kafka, Flink, and Spark.The role involves developing data pipelines, ensuring data quality, and efficiency. The successful...

  • Senior Data Engineering Specialist

    1 day ago

    Mysore, Karnataka, India beBeeEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Senior Data Engineering SpecialistWe are seeking a highly skilled and experienced Data Engineering Specialist to lead the development of our data pipeline architecture. As a key member of our team, you will be responsible for designing, building, and maintaining large-scale data pipelines using Apache Beam and Databricks.About the Role:The ideal...