Data Engineer-Azure databricks

2 weeks ago

Chennai, India Covenant Consultants Full time

We are seeking a highly skilled Azure Databricks Developer with relevant years of experience in Azure Cloud services and hands-on expertise with Databricks, Azure data factory,ADB, Python, and PySpark. The ideal candidate will be responsible for designing, implementing, and optimizing data processing pipelines and will play a critical role in our data engineering projects.

Key Responsibilities:

Data Engineering:

  • Design, develop, and optimize large-scale data processing pipelines using Azure Databricks, Azure data factory,etc.
  • Implement complex data transformations and processing workflows with PySpark.
  • Develop and maintain scalable data architecture and pipelines to support various data projects.

Azure Cloud Services:

  • Utilize various Azure services such as Azure Data Factory, Azure Data Lake Storage, and Azure Synapse Analytics to build end-to-end data solutions.
  • Monitor, troubleshoot, and optimize Azure cloud resources and Databricks clusters for performance and cost-efficiency.

Data Integration:

  • Integrate data from multiple sources and ensure data consistency and integrity.
  • Develop and implement data validation and quality checks.

Collaboration:

  • Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
  • Participate in agile development processes and contribute to continuous improvement practices.

Documentation and Reporting:

  • Document data workflows, architecture, and processes for knowledge sharing and maintenance.
  • Generate reports and visualizations to communicate insights and progress to stakeholders.

Required Skills and Experience:

Technical Skills:

  • Strong experience with Azure Databricks , Azure data factory,etc and PySpark.
  • Proficiency in Python programming, with a focus on data processing and manipulation.
  • Experience with Azure services such as Azure Data Factory, Azure Data Lake Storage, and Azure Synapse Analytics.
  • Solid understanding of data warehousing concepts and ETL processes.
  • Familiarity with CI/CD practices and tools for data pipelines.

Professional Experience:

  • Rel years of hands-on experience in data engineering and cloud-based data processing.
  • Proven track record of working with large-scale data systems and distributed computing.
  • Experience in performance tuning and optimization of data processing jobs.

Soft Skills:

  • Excellent problem-solving skills and attention to detail.
  • Strong communication skills to collaborate effectively with cross-functional teams.
  • Ability to work independently and manage multiple tasks in a dynamic environment.


  • Data Engineer

    4 months ago

    Chennai, India ResourceTree Global Services Full time

    Job Description :ResourceTree Global Services is hiring Senior Data Engineer Role for one of its Data Analytics Client. Below are the details Role : Senior Data Engineer - AzurePrimary Skills : ADF (Azure Data Factory), SQLExperience : 4 - 6 YearsLocation : ChennaiRole Responsibilities :Translate functional specifications and change requests into technical...

  • Data Engineer

    4 months ago

    Chennai, Tamil Nadu, India ResourceTree Global Services Full time

    Job Description :ResourceTree Global Services is hiring Senior Data Engineer Role for one of its Data Analytics Client. Below are the details Role : Senior Data Engineer - AzurePrimary Skills : ADF (Azure Data Factory), SQLExperience : 4 - 6 YearsLocation : ChennaiRole Responsibilities :Translate functional specifications and change requests into technical...

  • Data Engineer

    6 months ago

    Chennai, India ResourceTree Global Services Full time

    Job Description : ResourceTree Global Services is hiring Senior Data Engineer Role for one of its Data Analytics Client. Below are the details Role : Senior Data Engineer - AzurePrimary Skills : ADF (Azure Data Factory), SQLExperience : 4 - 6 Years Location : ChennaiRole Responsibilities : Translate functional specifications and change requests into...

  • Azure Databricks Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India Tata Consultancy Services Full time

    Job Title: Azure DatabricksGreetings from Tata Consultancy ServicesJob Title: Digital : Azure DatabricksExperience Range: 4-10 YearsMinimum Qualification:15 years of full-time educationJob Description:Key ResponsibilitiesDesign and develop data warehouse solutions using Azure DatabricksExperience with data platform architecture, deployment, and data...

  • Cloud Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Covenant Consultants Full time

    We are seeking a talented Azure Databricks Developer to lead our data engineering projects. As a key member of our team, you will design, implement, and optimize large-scale data processing pipelines using Azure Databricks, Azure Data Factory, and other related services.Key Responsibilities:Data Engineering:Design, develop, and optimize complex data...

  • Eucloid - Data Engineer - Azure Databricks

    6 months ago

    Gurgaon/Gurugram/Chennai, India Eucloid Data Solutions Pvt Ltd Full time

    Job Description - Data Engineer (Databricks)About Eucloid : Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for...

  • **Cloud Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Risk Resources LLP Full time

    Job Title: Azure Databricks SQL Snowflake SpecialistJob Summary: We are seeking a highly skilled Azure Databricks SQL Snowflake Specialist to join our team at Risk Resources LLP. The successful candidate will play a key role in designing, developing, and maintaining data solutions using Azure Databricks SQL and Snowflake. Key Responsibilities:Design and...

  • Senior Databricks Engineer

    3 weeks ago

    Bangalore/Chennai/Hyderabad, India Hirist Full time

    Job Description:We are seeking a skilled Senior Databricks Engineer to join our team at Hirist. As a key member of our Azure Data Services team, you will be responsible for designing, developing, and maintaining robust data pipelines using Databricks and Azure Data Factory.Key Responsibilities: Design and develop data pipelines using Databricks and ADF ...

  • Big Data Solutions Architect

    4 weeks ago

    Chennai, Tamil Nadu, India Databricks Full time

    Job Summary:As a Solutions Architect (Big Data) at Databricks, you will work with customers on short to medium-term customer engagements on their big data challenges using the Databricks platform. You will provide data engineering, data science, and cloud technology projects that require integrating with client systems, training, and other technical tasks to...

  • Big Data Migration Specialist

    3 weeks ago

    Chennai, Tamil Nadu, India Databricks Full time

    Job SummaryAs a Migrations Architect at Databricks, you will work with customers on short to medium-term engagements on their big data challenges using the Databricks platform. You will provide data engineering, data science, and cloud technology projects that require integrating with client systems, training, and other technical tasks to help customers get...

  • Senior Big Data Migration Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Databricks Full time

    About the RoleAs a Senior Big Data Migration Engineer at Databricks, you will work closely with customers on short to medium-term engagements on their big data challenges using our platform. Your expertise will be invaluable in providing data engineering, data science, and cloud technology projects that require integrating with client systems, training, and...

  • Azure Databricks

    4 weeks ago

    Chennai, India Tata Consultancy Services Full time

    Greetings from TCS!! Job Title: Digital : Azure Databricks location : Chennai Experience Range: 4-10 Years Minimum Qualification: 15 years of full-time education Job Description: Strong design and development experience in Data Ware House Experience with data platform architecture, deployment and data migration on Microsoft Azure platform ...

  • Eucloid - Lead Data Engineer - Python/Azure Databricks

    6 months ago

    Chennai, India Eucloid Data Solutions Pvt Ltd Full time

    Role : Lead Data Engineer (Databricks) About Eucloid : Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for some of...

  • Data Engineering Lead

    2 weeks ago

    Chennai, Tamil Nadu, India Databricks Full time

    We are seeking an experienced Data Engineering Lead to join our Professional Services team at Databricks. As a key member of our team, you will work closely with customers to deliver short to medium-term customer engagements on their big data challenges using the Databricks platform.Your responsibilities will include providing data engineering, data science,...

  • Data Migration Architect

    4 weeks ago

    Chennai, Tamil Nadu, India Databricks Full time

    Job Title: Resident Solution Architect, Migrations SpecialistAs a key member of our Professional Services team, you will work closely with customers on short to medium-term engagements to address their big data challenges using the Databricks platform. Your expertise will be instrumental in providing data engineering, data science, and cloud technology...

  • Senior Data Engineer

    4 months ago

    Chennai, Tamil Nadu, India Resource Tree Global Services Full time

    Details :Job Name : Senior Data Engineer - AzureYears of Experience : 4Job Description : We are looking for a skilled and experienced Senior Azure Developer to join our team! As part of the team, you will be involved in the implementation of the ongoing and new initiatives for our company. If you love learning, thinking strategically, innovating, and helping...

  • Senior Data Engineer

    4 months ago

    Chennai, India Resource Tree Global Services Full time

    Details :Job Name : Senior Data Engineer - AzureYears of Experience : 4Job Description : We are looking for a skilled and experienced Senior Azure Developer to join our team! As part of the team, you will be involved in the implementation of the ongoing and new initiatives for our company. If you love learning, thinking strategically, innovating, and helping...

  • Azure Data Engineer

    2 months ago

    Chennai, India Breezy Talent Full time

    Vidpro Consultancy Services is seeking a Senior Databricks Engineer / Databricks Technical Lead / Data Architect with 510 years of experience.Location: Bangalore Chennai Delhi Pune Kolkata and GurgaonWork Type: HybridMandatory Skills: Python/ PySpark / Spark with Azure/ Azure Databricks / Azure DataFactoryPrimary Roles and Responsibilities: Creating Modern...

  • Azure Data Engineer

    1 month ago

    chennai, India Breezy Talent Full time

    Vidpro Consultancy Services is seeking a Senior Databricks Engineer / Databricks Technical Lead / Data Architect with 510 years of experience.Location: Bangalore Chennai Delhi Pune Kolkata and GurgaonWork Type: HybridMandatory Skills: Python/ PySpark / Spark with Azure/ Azure Databricks / Azure DataFactoryPrimary Roles and Responsibilities: Creating Modern...

  • Azure Data Engineer

    3 weeks ago

    chennai, India Breezy Talent Full time

    Vidpro Consultancy Services is seeking a Senior Databricks Engineer / Databricks Technical Lead / Data Architect with 510 years of experience. Location: Bangalore Chennai Delhi Pune Kolkata and GurgaonWork Type: HybridMandatory Skills: Python/ PySpark / Spark with Azure/ Azure Databricks / Azure DataFactoryPrimary Roles and Responsibilities: Creating...