Unlock the Potential of Big Data: Senior Data Engineer Position

2 days ago

Shimoga, Karnataka, India beBeeDataEngineer Full time ₹ 1,30,17,368 - ₹ 2,09,28,996
Senior Data Engineer

This is a high-level position that requires in-depth knowledge of data engineering principles and practices.

The ideal candidate should have extensive experience with designing, building, and maintaining large-scale data pipelines using technologies such as Apache Spark and Python.

  • Key Responsibilities:
  • Design, build, and optimize complex data pipelines to process large datasets.
  • Manage and process real-time streaming data using Apache Spark Streaming/Kafka.
  • Collaborate with cross-functional teams to integrate AI/ML models into production.
  • Ensure data quality, scalability, and performance in all pipelines.
  • Deploy and manage Spark workloads on cloud platforms such as AWS, Azure, and GCP.
  • Automate testing and deployment of Spark jobs via CI/CD pipelines.

Required Skills and Qualifications

  • Bachelor's/Master's degree in Computer Science, Computer Engineering, or related field.
  • Minimum 4 years of professional experience as a Data Engineer.
  • Strong expertise in Python, Apache Spark, and PySpark.
  • Familiarity with data ingestion tools (Kafka, Flume, Kinesis) and data formats (Parquet, Avro, ORC).
  • Proficiency in Airflow/Prefect for scheduling and orchestration.
  • Familiarity with performance tuning in Spark (partitioning, caching, broadcast joins).
  • Experience deploying on Databricks, AWS EMR, Azure HDInsight, or GCP Dataproc.
  • Exposure to testing and CI/CD for data pipelines (pytest, Jenkins, GitHub Actions).

Benefits

  • Opportunity to work with cutting-edge technologies and innovative projects.
  • Collaborative and dynamic work environment.
  • Professional growth and development opportunities.

About the Role

This role offers an exciting opportunity to join a team of experienced data engineers and contribute to the design, development, and maintenance of our data infrastructure. We are looking for a highly skilled and motivated individual who can work effectively in a fast-paced environment and deliver high-quality results.

  • Big Data Expert

    4 days ago

    Shimoga, Karnataka, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    **Unlocking Big Data Opportunities:**We are seeking a highly skilled data expert to design and develop innovative solutions that drive business growth.Essential Skills:PySparkHiveDesirable Skills:SparkHBaseData Quality ToolsAgile Methodologies ExperienceExposure in Data Ingestion from Various Sources onto Big Data PlatformsRequirements:A minimum of 6-12...

  • Big Data Engineering Leader

    1 hour ago

    Shimoga, Karnataka, India beBeeDataEngineering Full time ₹ 1,80,00,000 - ₹ 2,40,00,000

    Big Data Engineering LeaderJob Title: Big Data Engineering LeaderLocation: Shivamogga, IndiaJob DescriptionWe are seeking a highly experienced leader to spearhead our data engineering efforts. The successful candidate will have a proven track record of leading cross-functional teams and working on large-scale distributed systems.Key Responsibilities:Leading...

  • Senior Data Engineer

    6 days ago

    Shimoga, Karnataka, India beBeeDataArchitect Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Data ArchitectUnlock Business Potential with Scalable Data SolutionsWe're a boutique consulting firm specializing in data and AI, with deep expertise in the Databricks ecosystem.Solutions that drive real business impact. We empower clients to harness the full potential of their data, unlocking significant competitive advantages and fostering innovation.Key...

  • Data Engineering Leader

    2 days ago

    Shimoga, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Senior Data EngineerThe role of a Senior Data Engineer involves designing, developing and maintaining data pipelines and data warehouse solutions. This position plays a critical role in ensuring the seamless flow of data across systems.This enables data-driven decision-making for our organization and clients.Key Responsibilities:Design scalable...

  • Senior Big Data Specialist

    6 days ago

    Shimoga, Karnataka, India beBeeBigDataSpecialist Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Senior Big Data SpecialistSkill Set:AWS - Amazon Web ServicesPythonPysparkAbout the Role:We are seeking a skilled big data specialist to join our team. The ideal candidate will have hands-on experience with AWS, Python, and Pyspark.Responsibilities:Built pySpark applications using Spark Dataframes in PythonOptimized spark jobs that process huge...

  • Big Data Engineering Solution Specialist

    4 days ago

    Shimoga, Karnataka, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Solution Architect for Big Data EngineeringWe are seeking an experienced Solution Architect to lead the design and implementation of large-scale data engineering solutions. The ideal candidate will have a strong background in Azure Data Engineering, with expertise in Data Factory, Synapse, Fabric, and Databricks. Additionally, they should have...

  • Senior Data Engineer

    1 hour ago

    Shimoga, Karnataka, India beBeeData Full time ₹ 20,00,000 - ₹ 30,00,000

    Job DescriptionThis is a challenging opportunity to leverage technical expertise in data warehousing, big data, and data analytics.The ideal candidate will have strong knowledge of Google Cloud Platform data components, including Big Query, BigTable, Cloud SQL, Data Flow, and Data Fusion.Required Skills and QualificationsKey Responsibilities:Maintain...

  • Big Data Architectural Specialist

    6 days ago

    Shimoga, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Transform Your Career as a Senior Data EngineerWe seek a highly skilled data professional to join our team of consultants.This role involves designing, constructing, and optimizing large-scale data processing systems that enable advanced analytics and business intelligence.About the Role:You will collaborate closely with data scientists, analysts, and other...

  • Unlock Your Potential as a Future Business Leader

    2 days ago

    Shimoga, Karnataka, India beBeeCareer Full time ₹ 3,00,00,000 - ₹ 4,00,00,000

    Unlock Your PotentialAs a high-potential graduate, you have the unique opportunity to join our fast-track journey and become a future Category Leader and Business Head.Through our 3-year program, you will gain hands-on experience in performance marketing, data analytics, D2C operations, marketplace management, and key account management.Our MissionWe aim to...

  • Big Data Specialist

    1 hour ago

    Shimoga, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Big Data SpecialistDuration: Full-time OpportunityLocation: Shivamogga, India (Work from office)Key Responsibilities:Design and develop large-scale data processing architectures using SQL, Python, PySpark, and Linux.Implement complex data pipelines and distributed systems to handle high-velocity data streams effectively.Collaborate with...