EverestDX - Senior Data Engineer - Python/PySpark

2 days ago


Hyderabad, Telangana, India EverestDX Inc Full time

Description
Job Summary :

  • We are seeking an experienced Azure Data Engineer / Lead with strong expertise in Azure services, distributed data processing, and team leadership.
  • The ideal candidate will have hands-on experience with Azure Databricks, Azure Data Lake, Azure Data Factory, PySpark/Spark, and SQL, along with the ability to guide a small team on technical design and delivery.

Key Responsibilities

  • Design, develop, and optimize large-scale data pipelines using Azure Databricks, ADLS, and ADF.
  • Implement and manage distributed data processing solutions using Spark/PySpark.
  • Write and optimize SQL queries for data extraction, transformation, and analysis.
  • Lead and mentor a small team of data engineers, providing technical guidance and reviewing design solutions.
  • Collaborate with stakeholders to translate business requirements into scalable and efficient data solutions.
  • Ensure data quality, governance, and compliance across data platforms.
  • Support production deployments, troubleshooting, and performance optimization.

Required Skills

  • Strong expertise with Azure Databricks, Azure Data Lake, and Azure Data Factory.
  • Hands-on experience in Spark/PySpark for distributed data processing.
  • Proficiency in SQL for complex queries and optimization.
  • Solid understanding of Azure cloud platform and its ecosystem.
  • Proven experience in leading and mentoring teams.
  • Strong problem-solving and analytical skills.
  • Excellent communication and collaboration abilities

)



  • Hyderabad, Telangana, India EverestDX Inc Full time

    Key ResponsibilitiesData Pipeline Development :Design, develop, and maintain end-to-end Azure Data Factory (ADF) pipelines for ingesting data from multiple structured and unstructured sources.Build scalable ETL/ELT processes to extract, transform, and load data into Azure Data Lake, Azure SQL Data Warehouse / Synapse, and other cloud storage...


  • Hyderabad, Telangana, India Rapsys Technologies Pte. Ltd Full time

    Role: Senior PySpark EngineerExperience Required: Minimum 8+ YearsWork Location: Hyderabad (5 Days Work from Office)Job Type: Contract to Hire (1 Year/ Renewable)Notice Period: Immediate to 15 Days maxMode of Interview: VirtualWe are seeking a highly skilled PySpark Data Engineer to design, build, and optimize large-scale data pipelines and distributed...


  • Hyderabad, Telangana, India EverestDX Inc Full time

    DescriptionAbout the Company :Everest DX We are a Digital Platform Services company, headquartered in Stamford.Our Platform/Solution includes Orchestration, Intelligent operations with BOTs, AI-powered analytics for Enterprise IT.Our vision is to enable Digital Transformation for enterprises to deliver seamless customer experience, business efficiency and...

  • PySpark Engineer

    2 weeks ago


    Hyderabad, Telangana, India Rapsys Technologies Pte. Ltd Full time ₹ 6,00,000 - ₹ 25,00,000 per year

    Role: PySpark EngineerExperience Required: Minimum 5-10 YearsWork Location: Hyderabad (5 Days Work from Office)Job Type: Contract to Hire (1 Year/ Renewable)Notice Period: Immediate to 30 Days maxRequired skill sets and expertise:5+ years of experience in PySpark, Big Data technologies, Data Warehousing.Strong Python programming experience.Domain: Experience...


  • Hyderabad, Telangana, India Egen Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Job Overview:We are looking for an experienced Senior Data Engineer to design, build, and optimize scalable, high-performance data platforms using AWS cloud services and Python. The ideal candidate will play a key role in architecting end-to-end data pipelines, driving automation, ensuring data quality, and enabling analytics and AI workloads across the...


  • Hyderabad, Telangana, India EverestDX Inc Full time

    DescriptionAbout the Company :Everest DX We are a Digital Platform Services company, headquartered in Stamford.Our Platform/Solution includes Orchestration, Intelligent operations with BOTs, AI-powered analytics for Enterprise IT.Our vision is to enable Digital Transformation for enterprises to deliver seamless customer experience, business efficiency and...


  • Hyderabad, Telangana, India People Prime Worldwide Full time

    Job Title:Data EngineerExperience:5–10 YearsLocation:PAN IndiaEmployment Type:Full-TimeJob DescriptionWe are seeking aData Engineerwith strong hands-on experience inPython, PySpark, Apache Spark, Kafka, and Databricksto build and maintain scalable, high-performance data pipelines. The role requires deep coding expertise and experience working with...


  • Hyderabad, Telangana, India EverestDX Inc Full time

    DescriptionWe are looking for a skilled and visionary Lead Cloud Automation Engineer to drive automation initiatives, lead a high-performing engineering team, and support the design, deployment, and optimization of cloud-based solutions.The ideal candidate is a hands-on technologist with experience in automation development, cloud architecture, CI/CD...

  • Senior Data Engineers

    2 weeks ago


    Hyderabad, Telangana, India SID Information Technologies Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    hiring #Hiring #hyderabad #hyderabadjob #HyderabadJobs #immediatejoiner #opportunity #CareerOpportunity #Senior #TechJobs #dataengineer #DataEngineering #DataArchitect #DataEngineeringJobs #HealthcareDataCloudComputing #BigData #DataAnalytics #PlatformEngineering #Terraform #Kubernetes #SQL #Python #Scala #apachespark #pyspark #ETLpipeline #PySpark...


  • Hyderabad, Telangana, India DATAECONOMY Full time

    Job Title: PySpark Data EngineerExperience: 6+ YearsLocation: Hyderabad/ PuneEmployment Type: Full-TimeJob Summary:We are looking for a skilled and experienced PySpark Data Engineer to join our growing data engineering team. The ideal candidate will have 6+ years of experience in designing and implementing data pipelines using PySpark, AWS Glue,...