Data Engineer(Pyspark+SQL)
2 weeks ago
About The Role
We are seeking an experienced Data Engineer to design, implement, and optimize a global data handling and synchronization solution across multiple regions. You will work with cloud-based databases, data lakes, and distributed systems, ensuring compliance with data residency and privacy requirements (e.g., GDPR).
Requirements
6+ years of experience as a Data Engineer in cloud environments (preferably Microsoft Azure).
Strong knowledge of Azure SQL, Data Lake, Data Factory, PySpark, and related services.
Experience in Spark Optimization.
Experience with distributed data architectures and data synchronization across regions.
Familiarity with data privacy, security, and compliance (GDPR, etc.).
Proficiency in Python, SQL, and ETL tools.
Excellent problem-solving and communication skills.
Hands-on and self-contributing.
Preferred
Experience with MS-SQL, Cosmos DB, Databricks, and event-driven architectures.
Knowledge of CI/CD and infrastructure-as-code (Azure DevOps, ARM/Bicep, Terraform).
Key Responsibilities
Design and implement data pipelines for global and regional data synchronization (Azure SQL, Data Lake, Data Factory, PySpark, etc.).
Develop solutions for secure handling of PII and non-PII data, ensuring compliance with GDPR and other regulations.
Build and optimize ETL processes for anonymization, transformation, global synchronization and distribution of data.
Collaborate with software architects and DevOps to integrate data flows with application logic and deployment pipelines.
Set up monitoring, alerting, and documentation for data processes within the existing frameworks.
Advise on best practices for data partitioning, replication, and schema evolution.
-
Job - Data Engineer(Pyspark+SQL)
1 week ago
Gurgaon, Haryana, India Accolite Full time ₹ 12,00,000 - ₹ 36,00,000 per yearGeneral Description Of RoleProfessionals in this group design and implement high-performance, scalable and optimized data solutions for large enterprise-wide data mining and processing. They are responsible for design and development of data flows and Big Data Platform deployment and implementation. Incumbents usually require expert knowledge in Databricks,...
-
ETL - Pyspark Testing - Strong SQL Professional
2 weeks ago
Gurgaon, Haryana, India IDESLABS PRIVATE LIMITED Full time ₹ 15,00,000 - ₹ 25,00,000 per yearAt least 6-8 yrs of experience in ETL Testing with Automation Testing Expert in database testing using SQL. Must have worked on Databricks and aware of Databricks related concepts Check the data source locations and formats, perform a data count, and verify that the columns and data types meet the requirements. Test the accuracy of the data, and its...
-
Python/Pyspark Developer
23 hours ago
Gurgaon, Haryana, India DGLiger Consulting Pvt ltd Full time ₹ 15,00,000 - ₹ 25,00,000 per yearLocation: Gurgaon ( Work from office) We are looking for a Python PySpark Developer with 34 years of experience, primarily focused on Python programming and automation using GitHub Actions. The ideal candidate will be responsible for developing scalable Python-based data workflows, writing PySpark scripts for large-scale data processing, and implementing...
-
Lead Data Engineer
5 days ago
Gurgaon, Haryana, India EXL Service Full time ₹ 20,00,000 - ₹ 25,00,000 per yearLead Data Engineer Immediate Hiring Alert – Lead Data Engineer (5 days, Work from Office) - Expert in Azure, Databricks, PySpark & SQL with a passion for leading teams Total Experience: 8 yrs Must Have:3 yrs in Azure Data Engineering 2 yrs in Databricks & PySpark Strong SQL expertise Team leadership (5 members) Excellent...
-
Python/Pyspark Developer
2 weeks ago
Gurgaon, Haryana, India DGLiger Consulting Pvt ltd Full time ₹ 20,00,000 - ₹ 25,00,000 per yearWe are looking for a skilled Python PySpark Developer with 34 years of experience in designing, developing, and maintaining big data solutions. The ideal candidate will have hands-on expertise in Python, PySpark, and data pipeline development, along with strong problem-solving skills and the ability to work in a collaborative environment. Key...
-
Data Engineer
1 week ago
Gurgaon, Haryana, India Impetus Full time ₹ 15,00,000 - ₹ 25,00,000 per yearOpen Location- Gurgaon and BangaloreJob Description4-7 years' experience working on Data engineering & ETL/ELT processes, data warehousing, and data lake implementation with Cloud servicesHands on experience in designing and implementing solutions like creating/deploying jobs, Orchestrating the job/pipeline and infrastructure configurationsExpertise in...
-
Lead Data Engineer(MDM)
2 weeks ago
Gurgaon, Haryana, India Accolite Full time ₹ 12,00,000 - ₹ 36,00,000 per yearAbout The RoleWe are looking for a Lead Data Engineer to architect and guide the design and implementation of a global data handling and synchronization platform. In addition to being hands-on, you will provide technical leadership to a small team of data engineers and advise on master data management (MDM) best practices, ensuring compliance with global...
-
Data Engineer
6 days ago
Gurgaon, Haryana, India Apps Associates Careers Full time ₹ 6,00,000 - ₹ 18,00,000 per yearRole: Data Engineer Skills:• years of experience as a Data engineer with extensive working experience using Pyspark, Advanced SQL, Snowflake, complex understanding of SQL and Skills, Spark, Snowflake and Glue.• AWS expertise (Azure, google will work too) - Lambda, Glue, S3 etc.• Experience in software development, CI/CD, Agile methodology•...
-
Lead Data Engineer
3 days ago
Gurgaon, Haryana, India WOW Softech Full time ₹ 20,00,000 - ₹ 25,00,000 per yearExp- 8-15 years3+ years of hands on experience in Azure Data Engineering development project2+ years of experience in Databricks development2+ years of experience in PySpark.2+ years of experience in SQLProficient in SQL2+ years of experience in leading at least a 5+ member team in data engineering deliveryMust be fluent in communicationRole &...
-
Senior Data Engineer
5 days ago
Gurgaon, Haryana, India EXL Service Full time ₹ 15,00,000 - ₹ 25,00,000 per yearSenior Data Engineer We are looking for talented professionals to join our growing team in key position as Senior Data Engineer Skills: PySpark, SQL, Hadoop, AWS, Python, Spark, Scala, Hive, GCP, NoSQL database (Redis, Valkey, OpenSearch) Experience: 5-10 years To apply, share your profile.