Python, Pyspark, ADB, ADF
3 days ago
Job Description We are seeking an experienced and motivated Senior Data Engineer with strong expertise in Python, PySpark, Azure Databricks, and Azure Data Factory (ADF). The ideal candidate will design and implement scalable data pipelines and play a key role in enabling data-driven decision-making by integrating data across various sources. Key Responsibilities: - Design, develop, and maintain ETL/ELT data pipelines using ADF, Databricks, and PySpark. - Develop scalable and reusable data processing solutions in Azure cloud environment. - Work with large-scale datasets from structured and unstructured data sources. - Optimize and tune PySpark jobs for performance and reliability. - Integrate and transform data across Azure services (Blob Storage, Data Lake, Synapse, etc.). - Collaborate with data scientists, analysts, and business stakeholders to understand data requirements. - Ensure data quality, security, and compliance throughout the data pipeline lifecycle. - Develop and maintain documentation for data workflows and pipelines. - Troubleshoot and resolve data pipeline issues efficiently. Required Skills: - 610 years of experience in data engineering or related field. - Strong programming skills in Python. - Extensive experience with PySpark for distributed data processing. - Hands-on experience in Azure Databricks and ADF. - Knowledge of Azure ecosystem (Data Lake, Blob Storage, Synapse, Key Vault, etc.). - Good understanding of data warehousing and data modeling concepts. - Experience working with CI/CD pipelines for data workflows. - Strong problem-solving skills and ability to work independently. - Excellent communication and stakeholder collaboration skills. Preferred Qualifications: - Microsoft Certified: Azure Data Engineer Associate or similar certifications. - Experience with delta lake, streaming data using structured streaming. - Exposure to DevOps, Terraform, or other IAC tools for Azure resource provisioning. - Familiarity with data governance, data quality frameworks, and RLS.
-
Databricks, SQL, PySpark, Python
3 days ago
Hyderabad, India Fusion Plus Solutions Full timeJob Description Roles and Responsibilities - 5+ years of experience on IT industry in Data Engineering & Data Analyst role. - 5 years of development experience using tool Databricks and PySpark, Python, SQL - Proficient in writing SQL queries including writing of windows functions - Good communication skills with analytical abilities in doing problem solving...
-
Python pyspark adb adf
3 days ago
Hyderabad, India Fusion Plus Solutions Full timeJob Description - experience in designing large data distribution, integration with service-oriented architecture and/or data warehouse solutions, Data Lake solution using Azure Databricks with large and multi-format data - ability to translate working solution into implementable working package using Azure platform - good understanding on Azure storage Gen2...
-
Lead_Databricks_Engineer_22552
3 weeks ago
Hyderabad, India DXC Technology Full timeJob Description: At least 5+ years’ of relevant hands on development experience as Azure Data Engineering role Proficient in Azure technologies like ADB, ADF, SQL(capability of writing complex SQL queries), ADB, PySpark, Python, Synapse, Delta Tables, Unity Catalog Hands on in Python, PySpark or Spark SQL Hands on in Azure Analytics and DevOps Taking...
-
Data Engineer
16 hours ago
Hyderabad, Telangana, India Techno Facts Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per yearJob Description:We are seeking a highly skilled and motivated Data Engineer with 45 years of hands-on experience in building scalable data pipelines and transforming complex datasets. The ideal candidate should have strong expertise in Azure Data Factory (ADF), Azure Databricks (ADB), Python, and PySpark, along with solid proficiency in SQL.Key...
-
Hyderabad, India Fusion Plus Solutions Full timeJob Description Skill Set:Azure Data Factory, Azure Databricks, Python/Pyspark - Joining time ( Notice period) Immediate joiners only - AnAzure Data Factory, Azure Databricks, Python/PySpark Professional for Testingplays a vital role in ensuring the accuracy, performance, and reliability of data workflows that involve Azure Data Factory (ADF), Azure...
-
Python pyspark
3 weeks ago
Hyderabad, India Virtusa Full timePython pyspark - CREQ Description Design and develop on Hadoop applications. Hands on in developing Jobs in pySpark with Python/ SCALA (Preferred) or Java/ SCALA. Experience on Core Java, Experience on Map Reduce programs, Hive programming, Hive queries performance concepts. Experience on source code management with Git repositories. Secondary skills...
-
Hyderabad, India Fusion Plus Solutions Full timeJob Description Job Responsibilities: - Design, develop, and implement robust and scalable data pipelines using Azure Data Factory (ADF). - Efficiently extract, transform, and load (ETL) data from various disparate sources into Azure Data Lake Storage (ADLS). - Utilize Azure Databricks for advanced data processing, transformation, and analytics. - Develop...
-
Azure Data Engineer
2 weeks ago
Hyderabad, India Tata Consultancy Services Full timeTCS presents an excellent Opportunity For "Azure Data Engineer" RoleJob Description1 Role - Azure Data Engineer2 Location of Requirement - PAN India3 Experience Range - 5 to 10 YearsDesired Competencies- Azure Databricks - Basic understanding of ADB. knowledge of pyspark coding to create/debug the notebook.- ADF - Basic understanding of pipeline and all the...
-
Azure Data Engineer
1 week ago
Hyderabad, India Tata Consultancy Services Full timeTCS presents an excellent Opportunity For "Azure Data Engineer" Role Job Description 1 Role - Azure Data Engineer 2 Location of Requirement - PAN India 3 Experience Range - 6 to 8 Years Desired Competencies Azure Databricks - Basic understanding of ADB. knowledge of pyspark coding to create/debug the notebook. ADF - Basic understanding of pipeline and all...
-
Azure Data Engineer
2 weeks ago
Hyderabad, India Tata Consultancy Services Full timeTCS presents an excellent Opportunity For "Azure Data Engineer" Role Job Description 1 Role - Azure Data Engineer 2 Location of Requirement - PAN India 3 Experience Range - 5 to 10 Years Desired Competencies - Azure Databricks - Basic understanding of ADB. knowledge of pyspark coding to create/debug the notebook. - ADF - Basic understanding of pipeline and...
