Senior Data Engineer

2 weeks ago

Varanasi, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 18,00,000

Job Overview

This is a contract remote position for a highly motivated Data Engineer with 3–4 years of experience in building, optimizing, and maintaining large-scale data processing solutions.

The ideal candidate will bring hands-on expertise in PySpark, Python, SQL, and real-time data streaming and contribute to the design and implementation of scalable data pipelines in cloud environments such as Azure.

Key Responsibilities:

  • Data Pipeline Development: Design, develop, and optimize data pipelines using PySpark, Spark, and Python.
  • ETL/ELT Solutions: Implement ETL/ELT solutions to integrate structured and unstructured data from multiple sources.
  • Hive Queries: Develop and maintain Hive queries and ensure seamless data transformations.
  • SQL Query Optimization: Write and optimize SQL queries for large datasets.
  • Real-Time Streaming: Build, monitor, and support real-time streaming solutions using Apache Kafka.
  • Databricks Management: Work on Databricks to create, manage, and optimize data workflows.
  • Azure Cloud Deployment: Deploy and manage solutions in Azure cloud services (e.g., Data Lake, Synapse, Azure Databricks).
  • Code Debugging and Optimization: Debug, troubleshoot, and optimize code for performance and scalability.
  • Cross-Functional Collaboration: Collaborate with cross-functional teams to deliver data-driven solutions.

Required Skills & Qualifications:

  • Experience: 3–4 years of experience in data engineering with a focus on PySpark and Python.
  • Knowledge: Strong knowledge of Spark, Hive, SQL, and Unix shell scripting.
  • Databricks Experience: Experience with Databricks in a production environment.
  • Azure Cloud Experience: Hands-on experience with Azure cloud services (preferred: Data Lake, Synapse, Databricks).
  • Real-Time Streaming Experience: Experience working with Apache Kafka or other real-time streaming technologies.
  • Problem-Solving Skills: Strong problem-solving and debugging skills.
  • Communication Skills: Excellent written and verbal communication skills.

Nice-to-Have Skills:

  • CI/CD Pipelines: Exposure to CI/CD pipelines for data solutions.
  • Data Governance: Knowledge of data governance and security best practices.
  • Agile Methodology: Experience with Agile/Scrum methodology.

  • Senior Engineer-Mechanical

    2 days ago

    Varanasi, Uttar Pradesh, India ALP Talent Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Title: Senior Engineer- Mechanical (Planning/Billing)Position OverviewWe are seeking an experienced Senior Engineer to lead the fabrication drawings team forheavy steel structures. The role involves managing detailing activities, ensuring drawingaccuracy, optimizing material usage, and coordinating with production and project teams.Key RequirementsB.E. /...

  • Senior Data Scientist

    2 weeks ago

    Varanasi, Uttar Pradesh, India Perfios Full time

    Primary Function:-As a Senior Data Scientist, you will:● Work collaboratively with the engineering and product teams.● Communicate and articulate complex issues & technologies easily to stakeholders at different levels.● Think through problems and drive solutions from inception to shipment.● Keeping abreast of research trends in the field of Deep...

  • Sensor & Machine Vision Engineer

    3 weeks ago

    Varanasi, Uttar Pradesh, India Pando Data Full time

    Sensor & Machine Vision EngineerAt Pando Data, we're on a mission to bring Industry 4.0 to life in factories that need it most. We design smart, affordable AI + sensor systems that actually work on the shop floor - from measuring steel bars flying out of a rolling mill, to spotting defects in plastic pipes, to preventing theft in dispatch yards.We're a...

  • Senior Backend Engineer

    4 weeks ago

    Varanasi, Uttar Pradesh, India Rayole Software Full time

    We're Hiring: Senior Backend Engineer Location: RemoteEmployment Type: Full-TimeExperience Level: SeniorAre you an experienced Backend Engineer with a passion for building scalable, high-performance applications? We're looking for a highly skilled Senior Backend Engineer with extensive expertise in Python, Django, Next.js, Node.js, React, and web...

  • Senior Data Insights Specialist

    2 weeks ago

    Varanasi, Uttar Pradesh, India beBeeDataAnalytics Full time US$ 1,00,000 - US$ 1,50,000

    Key Role: Senior Analytics ConsultantOur team is seeking a highly skilled Senior Analytics Consultant to leverage advanced data analytics, predictive modeling, and real-world data to deliver high-impact outcomes for pharmaceutical companies, healthcare providers, and governments worldwide.This role involves leveraging various data sources such as claims,...

  • Large Scale Data Architect

    2 weeks ago

    Varanasi, Uttar Pradesh, India beBeeData Full time ₹ 45 - ₹ 55

    Job Title: Senior Data Engineer with GCP, ETLAbout the RoleWe are seeking a highly skilled Senior Data Engineer to design and implement large-scale data pipelines on Google Cloud Platform (GCP) using ETL tools and technologies.Key Responsibilities:Design and develop ETL pipelines on GCP using Python or JavaImplement data ingestion, processing, and storage...

  • Data Engineer

    3 weeks ago

    Varanasi, Uttar Pradesh, India MacroHire Full time

    Job Details-Designation - MicroSoft Fabric Lead (Data & AI- Data)Type of employment- Contract 6 months (Extendable)Rate- $16 per hourShift- ISTLocation- RemoteJD-Fabric Lead should have 9-12 years of experience, with 5+ years' experience in Azure Data Platform (ADF, Databricks, Synapse) and at least 1 year in MS Fabric (client implementation, not POC)...

  • Senior Data Scientist

    3 weeks ago

    Varanasi, Uttar Pradesh, India Innovaccer Full time

    Innovaccer is looking for aSenior Data ScientistHello You've landed on this page, which means you're interested in working with us. Let's take a sneak peek at what it's like to work at Innovaccer.Engineering at Innovaccer With every line of code, we accelerate our customers' success, turning complex challenges into innovative solutions. Collaboratively, we...

  • Career Opportunities as a Data Engineer

    2 weeks ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time US$ 1,00,000 - US$ 1,40,000

    Unlock Career Opportunities as a Data EngineerJob OverviewWe are seeking an experienced Data Engineer to join our team and drive data-driven decision making.About the RoleData Modeling and Engineering: Design, build, and maintain scalable and reusable data models using tools like dbt (Data Build Tool).Data Ingestion and Transformation: Collaborate with data...

  • Lead Data Engineer

    4 weeks ago

    Varanasi, Uttar Pradesh, India The Conqueror Challenges Full time

    About the RoleWe're looking for a relentlessly curious Data Engineer to join our team as a Marketing Platform Specialist. This role is for someone who will dedicate themselves to extracting every possible byte of data from marketing platforms - the kind of person who gets excited about discovering hidden API endpoints and undocumented features.. You'll...