Python, Pyspark, ADB, ADF
4 weeks ago
Job Description We are seeking an experienced and motivated Senior Data Engineer with strong expertise in Python, PySpark, Azure Databricks, and Azure Data Factory (ADF). The ideal candidate will design and implement scalable data pipelines and play a key role in enabling data-driven decision-making by integrating data across various sources. Key Responsibilities: - Design, develop, and maintain ETL/ELT data pipelines using ADF, Databricks, and PySpark. - Develop scalable and reusable data processing solutions in Azure cloud environment. - Work with large-scale datasets from structured and unstructured data sources. - Optimize and tune PySpark jobs for performance and reliability. - Integrate and transform data across Azure services (Blob Storage, Data Lake, Synapse, etc.). - Collaborate with data scientists, analysts, and business stakeholders to understand data requirements. - Ensure data quality, security, and compliance throughout the data pipeline lifecycle. - Develop and maintain documentation for data workflows and pipelines. - Troubleshoot and resolve data pipeline issues efficiently. Required Skills: - 610 years of experience in data engineering or related field. - Strong programming skills in Python. - Extensive experience with PySpark for distributed data processing. - Hands-on experience in Azure Databricks and ADF. - Knowledge of Azure ecosystem (Data Lake, Blob Storage, Synapse, Key Vault, etc.). - Good understanding of data warehousing and data modeling concepts. - Experience working with CI/CD pipelines for data workflows. - Strong problem-solving skills and ability to work independently. - Excellent communication and stakeholder collaboration skills. Preferred Qualifications: - Microsoft Certified: Azure Data Engineer Associate or similar certifications. - Experience with delta lake, streaming data using structured streaming. - Exposure to DevOps, Terraform, or other IAC tools for Azure resource provisioning. - Familiarity with data governance, data quality frameworks, and RLS.
-
Chennai, Hyderabad, Pune, India Tata Consultancy Services Full time ₹ 8,00,000 - ₹ 24,00,000 per yearTCS Hiring for skill "ADF, ADB, SQL, Pyspark, python".Role: Azure Data EngineerRequired Technical Skill Set: ADLS/ADF/ADB/Key VaultExperience: 8-12 YearsWork Location: Kolkata, Bangalore, Hyderabad, Chennai, Indore, Noida, Pune, MumbaiMust-Have:1.Lead Back-end development & maintenance of Data Quality ProductDesign and develop Data Pipeline using ADF,...
-
Databricks, SQL, PySpark, Python
4 weeks ago
Hyderabad, India Fusion Plus Solutions Full timeJob Description Roles and Responsibilities - 5+ years of experience on IT industry in Data Engineering & Data Analyst role. - 5 years of development experience using tool Databricks and PySpark, Python, SQL - Proficient in writing SQL queries including writing of windows functions - Good communication skills with analytical abilities in doing problem solving...
-
Python pyspark adb adf
4 weeks ago
Hyderabad, India Fusion Plus Solutions Full timeJob Description - experience in designing large data distribution, integration with service-oriented architecture and/or data warehouse solutions, Data Lake solution using Azure Databricks with large and multi-format data - ability to translate working solution into implementable working package using Azure platform - good understanding on Azure storage Gen2...
-
Data Engineer
5 days ago
hyderabad, India Techno Facts Solutions Full timeJob Description:We are seeking a highly skilled and motivated Data Engineer with 45 years of hands-on experience in building scalable data pipelines and transforming complex datasets. The ideal candidate should have strong expertise in Azure Data Factory (ADF), Azure Databricks (ADB), Python, and PySpark, along with solid proficiency in SQL.Key...
-
Senior Azure Data Engineer
5 days ago
Hyderabad, Kolkata, Noida, India Launchio Consulting Full time ₹ 12,00,000 - ₹ 36,00,000 per yearJob Title: Azure Data Engineer (ADF, Databricks, SQL)Experience: 8 to 12 YearsLocation: PAN India AvailableNotice Period: Immediate to 30 DaysEmployment Type: Full-timeJob Summary:We are seeking a skilled and experienced Azure Data Engineer to join our growing team. The ideal candidate should have a strong background in Azure Data Factory (ADF), Azure...
-
Python pyspark
2 days ago
Hyderabad, India Virtusa Full timePython pyspark - CREQ Description Design and develop on Hadoop applications. Hands on in developing Jobs in pySpark with Python/ SCALA (Preferred) or Java/ SCALA. Experience on Core Java, Experience on Map Reduce programs, Hive programming, Hive queries performance concepts. Experience on source code management with Git repositories. Secondary skills...
-
Hyderabad, India Fusion Plus Solutions Full timeJob Description Skill Set:Azure Data Factory, Azure Databricks, Python/Pyspark - Joining time ( Notice period) Immediate joiners only - AnAzure Data Factory, Azure Databricks, Python/PySpark Professional for Testingplays a vital role in ensuring the accuracy, performance, and reliability of data workflows that involve Azure Data Factory (ADF), Azure...
-
Senior Python Azure developer with PySparks
2 days ago
Hyderabad, India ScaleneWorks Full timeJob Title: Senior Node.js and Python Azure developer ( AWS to Azure Migration expert) Experience: 7-10 Yrs. Primary Skills: Python, PySpark - Python API Hands-on experience with Azure, Serverless (Azure Functions) AWS to Azure Cloud Migration (Preferred) Nice to have: Experience in Databricks , ADF mapping dataflow and Synapse Spark Scope of Work: Hand-on...
-
Hyderabad, India Fusion Plus Solutions Full timeJob Description Job Responsibilities: - Design, develop, and implement robust and scalable data pipelines using Azure Data Factory (ADF). - Efficiently extract, transform, and load (ETL) data from various disparate sources into Azure Data Lake Storage (ADLS). - Utilize Azure Databricks for advanced data processing, transformation, and analytics. - Develop...
-
Senior Data Engineer
2 weeks ago
Hyderabad, India Digivance Solutions Full timeJob Title: Data Engineer - Location: Hyderabad - Experience Required: 7+ years (3+ years relevant in PySpark, Azure Data Factory, Apache HIve and Python)Must have - Open Positions: 3 - Round of Interviews: 1- Assessment (1:30 hours) 2- Panel (Hyderabad F2F) - Valid Passport: Mandate Key Responsibilities - Design, develop, and maintain scalable data pipelines...
