Senior Data Engineer Position

12 hours ago

Varanasi, Uttar Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000
Big Data Engineer Role Summary

The role requires a highly skilled Big Data Engineer with expertise in Scala, Spark, Python, NiFi, and Apache Kafka.

Key responsibilities include designing, developing, and optimizing large-scale data pipelines and distributed data processing systems.

Additionally, the ideal candidate will have hands-on experience in streaming solutions using Apache Kafka and orchestration tools like NiFi/Airflow.

A strong grasp of software engineering concepts, data modeling, and optimization techniques is also necessary.

Required skills include Scala (minimum 3+ years), strong expertise in Spark/PySpark, and Python. Additionally, knowledge of NiFi/Airflow for orchestration, strong experience in Distributed Data Systems (5+ years), and proficiency in SQL and query optimization are required.

Preferred skills include exposure to messaging technologies like Apache Kafka or equivalent, experience in designing intuitive UIs for data analytics visualization, familiarity with Splunk or other monitoring/alerting solutions, and hands-on experience with CI/CD tools (Git, Jenkins).

Job Description:

We seek a seasoned Big Data Engineer to join our team.

Responsibilities:

  • Design, develop, and optimize large-scale data pipelines and distributed data processing systems.
  • Work extensively with Scala, Spark/PySpark, and Python for data processing and transformation.
  • Develop and integrate streaming solutions using Apache Kafka and orchestration tools like NiFi/Airflow.
  • Write efficient queries and perform data analysis using Jupyter Notebooks and SQL.
  • Collaborate with cross-functional teams to design scalable cloud-based data architectures.
  • Ensure delivery of high-quality code through code reviews, performance tuning, and best practices.
  • Build monitoring and alerting systems leveraging Splunk or equivalent tools.
  • Participate in CI/CD workflows using Git, Jenkins, and other DevOps tools.
  • Contribute to product development with a focus on scalability, maintainability, and performance.

Mandatory Skills:

  • Scala - Minimum 3+ years of hands-on experience.
  • Strong expertise in Spark/PySpark and Python.
  • Hands-on experience with Apache Kafka.
  • Knowledge of NiFi/Airflow for orchestration.
  • Strong experience in Distributed Data Systems (5+ years).
  • Proficiency in SQL and query optimization.
  • Good understanding of Cloud Architecture.

Preferred Skills:

  • Exposure to messaging technologies like Apache Kafka or equivalent.
  • Experience in designing intuitive UIs for data analytics visualization.
  • Familiarity with Splunk or other monitoring/alerting solutions.
  • Hands-on experience with CI/CD tools (Git, Jenkins).

  • Senior AI/ML Software Developer

    21 minutes ago

    Varanasi, Uttar Pradesh, India beBeeArtificial Intelligence Engineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    AI/ML Engineer – Senior ConsultantThe AI Engineering Group is part of the Data Science & AI Competency Center, focusing on technical and engineering aspects of DS/ML/AI solutions.We are seeking an experienced AI/ML Engineer to join our team and help bring AI/ML solutions into production, automate processes, and define reusable best practices and...

  • Data Architect Position

    2 days ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 3,00,00,000

    Senior Data ArchitectWe are seeking a highly skilled Senior Data Engineer to join our growing team of consultants. This is a remote position with a contract duration of 1 year, potentially extendable.In this role, you will be responsible for designing, constructing, and optimizing large-scale data processing systems that enable advanced analytics and...

  • Data Engineer Position

    8 hours ago

    Varanasi, Uttar Pradesh, India beBeeData Full time US$ 80,000 - US$ 1,40,000

    Job Title:Data Engineering SpecialistWe are seeking a skilled Data Engineer to join our dynamic team. As a key member of our data infrastructure team, you will design, develop, and maintain robust data pipelines, data warehouses, and analytics platforms.In this role, you will collaborate with cross-functional teams to ensure seamless data flow from various...

  • Senior IoT Data Engineer

    2 days ago

    Varanasi, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Senior IoT Data EngineerWe are seeking an experienced Senior IoT Data Engineer with hands-on coding expertise in Python and PySpark. The successful candidate will have a strong understanding of IoT systems, especially in refrigeration units, HVAC systems, controllers, and sensors.Key Responsibilities: Implement an application for a large retailer...

  • Data Engineering Lead

    3 days ago

    Varanasi, Uttar Pradesh, India beBeedata Full time ₹ 15,91,059 - ₹ 21,47,717

    Job Title: Senior Data EngineerWe are seeking a highly skilled and experienced Informatica developer to lead our data engineering team.

  • Senior Data Pipeline Specialist

    1 day ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 27,00,000

    Meet the requirements for a senior data engineer position to create and maintain ETL/ELT pipelines in our organization.Key ResponsibilitiesPipeline Development: Design, build, and manage efficient ETL/ELT pipelines using AWS services like Lambda, DMS, and Glue.Data Transformation: Implement complex business logic and data transformations utilizing Snowflake...

  • Senior Marketing Data Strategist

    2 days ago

    Varanasi, Uttar Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,40,00,000

    Marketing Data Governance Manager">As a Marketing Data Governance Manager, you will play a pivotal role in shaping our company's data strategy. This key position partners with global and regional leaders to analyze, document, optimize, and orchestrate enterprise data for marketing and consumer engagement.This role requires expertise in data analysis,...

  • Senior Data Strategist

    1 day ago

    Varanasi, Uttar Pradesh, India beBeeDataStrategist Full time ₹ 2,50,00,000 - ₹ 3,00,00,000

    About the RoleThis is a senior-level position within our data organization, requiring extensive experience in technical, solutioning, and analytical roles. The ideal candidate will have 10+ years of experience in building and managing data lakes, data warehouses, data integration, data migration, and business intelligence/artificial intelligence solutions on...

  • Lead Data Solutions Engineer

    1 day ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineering Full time US$ 1,80,000 - US$ 2,50,000

    Our Senior Data Engineering team is seeking a seasoned professional to design and optimize data solutions. The ideal candidate will be instrumental in designing and maintaining scalable data pipelines using Snowflake, Matillion, and Python.The role requires deep expertise in ETL development, data modeling, and financial data domains, particularly within...

  • Senior Geographic Data Engineer

    1 day ago

    Varanasi, Uttar Pradesh, India beBeeGeospatial Full time ₹ 8,00,000 - ₹ 12,00,000

    Geospatial Data SpecialistThis position plays a pivotal role in developing and managing geospatial data across various projects. Key responsibilities include creating, digitizing, and coding GIS data as well as collecting and processing spatial data to ensure accuracy and integrity.Key Responsibilities:Create and manage GIS data including digitization and...