EverestDX - Senior Data Engineer - Python/PySpark
2 days ago
Description
Job Summary :
- We are seeking an experienced Azure Data Engineer / Lead with strong expertise in Azure services, distributed data processing, and team leadership.
- The ideal candidate will have hands-on experience with Azure Databricks, Azure Data Lake, Azure Data Factory, PySpark/Spark, and SQL, along with the ability to guide a small team on technical design and delivery.
Key Responsibilities
- Design, develop, and optimize large-scale data pipelines using Azure Databricks, ADLS, and ADF.
- Implement and manage distributed data processing solutions using Spark/PySpark.
- Write and optimize SQL queries for data extraction, transformation, and analysis.
- Lead and mentor a small team of data engineers, providing technical guidance and reviewing design solutions.
- Collaborate with stakeholders to translate business requirements into scalable and efficient data solutions.
- Ensure data quality, governance, and compliance across data platforms.
- Support production deployments, troubleshooting, and performance optimization.
Required Skills
- Strong expertise with Azure Databricks, Azure Data Lake, and Azure Data Factory.
- Hands-on experience in Spark/PySpark for distributed data processing.
- Proficiency in SQL for complex queries and optimization.
- Solid understanding of Azure cloud platform and its ecosystem.
- Proven experience in leading and mentoring teams.
- Strong problem-solving and analytical skills.
- Excellent communication and collaboration abilities
)
-
EverestDX - Azure Data Engineer
1 week ago
Hyderabad, Telangana, India EverestDX Inc Full timeKey ResponsibilitiesData Pipeline Development :Design, develop, and maintain end-to-end Azure Data Factory (ADF) pipelines for ingesting data from multiple structured and unstructured sources.Build scalable ETL/ELT processes to extract, transform, and load data into Azure Data Lake, Azure SQL Data Warehouse / Synapse, and other cloud storage...
-
Senior PySpark Engineer
1 day ago
Hyderabad, Telangana, India Rapsys Technologies Pte. Ltd Full timeRole: Senior PySpark EngineerExperience Required: Minimum 8+ YearsWork Location: Hyderabad (5 Days Work from Office)Job Type: Contract to Hire (1 Year/ Renewable)Notice Period: Immediate to 15 Days maxMode of Interview: VirtualWe are seeking a highly skilled PySpark Data Engineer to design, build, and optimize large-scale data pipelines and distributed...
-
Hyderabad, Telangana, India EverestDX Inc Full timeDescriptionAbout the Company :Everest DX We are a Digital Platform Services company, headquartered in Stamford.Our Platform/Solution includes Orchestration, Intelligent operations with BOTs, AI-powered analytics for Enterprise IT.Our vision is to enable Digital Transformation for enterprises to deliver seamless customer experience, business efficiency and...
-
PySpark Engineer
2 weeks ago
Hyderabad, Telangana, India Rapsys Technologies Pte. Ltd Full time ₹ 6,00,000 - ₹ 25,00,000 per yearRole: PySpark EngineerExperience Required: Minimum 5-10 YearsWork Location: Hyderabad (5 Days Work from Office)Job Type: Contract to Hire (1 Year/ Renewable)Notice Period: Immediate to 30 Days maxRequired skill sets and expertise:5+ years of experience in PySpark, Big Data technologies, Data Warehousing.Strong Python programming experience.Domain: Experience...
-
Senior Data Engineer – Python
2 weeks ago
Hyderabad, Telangana, India Egen Full time ₹ 12,00,000 - ₹ 24,00,000 per yearJob Overview:We are looking for an experienced Senior Data Engineer to design, build, and optimize scalable, high-performance data platforms using AWS cloud services and Python. The ideal candidate will play a key role in architecting end-to-end data pipelines, driving automation, ensuring data quality, and enabling analytics and AI workloads across the...
-
Hyderabad, Telangana, India EverestDX Inc Full timeDescriptionAbout the Company :Everest DX We are a Digital Platform Services company, headquartered in Stamford.Our Platform/Solution includes Orchestration, Intelligent operations with BOTs, AI-powered analytics for Enterprise IT.Our vision is to enable Digital Transformation for enterprises to deliver seamless customer experience, business efficiency and...
-
Senior Data Engineer
1 day ago
Hyderabad, Telangana, India People Prime Worldwide Full timeJob Title:Data EngineerExperience:5–10 YearsLocation:PAN IndiaEmployment Type:Full-TimeJob DescriptionWe are seeking aData Engineerwith strong hands-on experience inPython, PySpark, Apache Spark, Kafka, and Databricksto build and maintain scalable, high-performance data pipelines. The role requires deep coding expertise and experience working with...
-
Hyderabad, Telangana, India EverestDX Inc Full timeDescriptionWe are looking for a skilled and visionary Lead Cloud Automation Engineer to drive automation initiatives, lead a high-performing engineering team, and support the design, deployment, and optimization of cloud-based solutions.The ideal candidate is a hands-on technologist with experience in automation development, cloud architecture, CI/CD...
-
Senior Data Engineers
2 weeks ago
Hyderabad, Telangana, India SID Information Technologies Full time ₹ 8,00,000 - ₹ 24,00,000 per yearhiring #Hiring #hyderabad #hyderabadjob #HyderabadJobs #immediatejoiner #opportunity #CareerOpportunity #Senior #TechJobs #dataengineer #DataEngineering #DataArchitect #DataEngineeringJobs #HealthcareDataCloudComputing #BigData #DataAnalytics #PlatformEngineering #Terraform #Kubernetes #SQL #Python #Scala #apachespark #pyspark #ETLpipeline #PySpark...
-
Lead/ Sr. Pyspark Data Engineer
1 day ago
Hyderabad, Telangana, India DATAECONOMY Full timeJob Title: PySpark Data EngineerExperience: 6+ YearsLocation: Hyderabad/ PuneEmployment Type: Full-TimeJob Summary:We are looking for a skilled and experienced PySpark Data Engineer to join our growing data engineering team. The ideal candidate will have 6+ years of experience in designing and implementing data pipelines using PySpark, AWS Glue,...