Data Engineer

2 months ago

Chennai, India Revature Full time
About the RoleWe are looking for a skilled Data Engineering Analyst with expertise in PySpark, Python, Hadoop, and Azure Databricks to join our team. You will play a key role in data integration, transformation, and analysis, working with large datasets to deliver insights that drive business decisions. In this role, you will collaborate with cross-functional teams, including data scientists, data engineers, and business analysts, to develop and maintain data pipelines, and ensure data quality and consistency across systems.Key ResponsibilitiesDesign, develop, and optimize ETL pipelines for data ingestion, transformation, and loading, using PySpark and Azure Databricks.Implement data processing and transformation logic in Python and PySpark to handle large datasets efficiently.Work with Hadoop-based data lakes and Azure cloud storage solutions to manage and retrieve data as required.Collaborate with data scientists and analysts to provide data-driven insights and solutions.Perform data profiling, validation, and quality assurance to ensure data accuracy and consistency.Optimize data models and pipelines for performance, scalability, and reliability.Utilize Azure Databricks for data processing workflows, managing resources, and ensuring job efficiency.Monitor, troubleshoot, and resolve data pipeline issues to ensure minimal downtime and data accuracy.Contribute to documentation and best practices for data engineering processes and standards.Required Skills and QualificationsEducation : Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related field.Experience : 3+ years of experience in data engineering, with hands-on experience in PySpark, Python, Hadoop, and Azure Databricks.Technical Skills :Strong experience with

Python

and

PySpark

for data transformation and analysis.Proficiency in

Hadoop

ecosystems (HDFS, Hive, HBase) for data storage and retrieval.Experience with

Azure Databricks

and other Azure data services, including Azure Data Lake, Azure Blob Storage, and Azure Synapse.Strong SQL skills for data manipulation, querying, and validation.Knowledge of ETL processes and tools, including performance optimization techniques.Analytical Skills :Strong problem-solving skills with the ability to troubleshoot complex data issues.Proficiency in handling large datasets and performing data profiling and cleansing.Soft Skills :Excellent communication and teamwork abilities to collaborate with cross-functional teams.Strong attention to detail and ability to work independently and manage multiple tasks effectively.

  • Data Engineering Director

    7 days ago

    Chennai, Tamil Nadu, India Wavicle Data Solutions Full time

    OverviewWelcome to Wavicle Data Solutions, a leading provider of innovative data management solutions. We are seeking an experienced Data Engineering Director to join our team and lead the development of our data engineering initiatives.

  • Lead Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Wavicle Data Solutions Full time

    Job Title: Lead Data Engineer**Overview**We are seeking an experienced Lead Data Engineer to join our team at Wavicle Data Solutions. The successful candidate will play a pivotal role in architecting and implementing data solutions, leveraging their expertise in managing data-related projects, including Data Engineering and Data Analytics...

  • Senior Machine Learning Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India Crayon Data Full time

    We are seeking a highly skilled Senior Machine Learning Engineer to join our team in Chennai. This is a fantastic opportunity to work on large scale data solutions using machine learning, operation research and pattern searching methods.About Crayon DataCrayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore...

  • Bastion Data Solutions | Cloud SRE Architect

    4 weeks ago

    Chennai, India Bastion Data Solutions Full time

    Company DescriptionBastion data solutions..Role DescriptionThis is a full-time on-site role for a Cloud SRE Architect / DevOps Engineer at Bastion Data Solutions in Chennai. The role involves day-to-day tasks such as Site Reliability Engineering, troubleshooting, software development, system administration, and infrastructure management.QualificationsSite...

  • Bastion Data Solutions | Cloud SRE Architect

    4 weeks ago

    chennai, India Bastion Data Solutions Full time

    Company DescriptionBastion data solutions..Role DescriptionThis is a full-time on-site role for a Cloud SRE Architect / DevOps Engineer at Bastion Data Solutions in Chennai. The role involves day-to-day tasks such as Site Reliability Engineering, troubleshooting, software development, system administration, and infrastructure management.QualificationsSite...

  • Bastion Data Solutions | Cloud SRE Architect

    1 month ago

    chennai, India Bastion Data Solutions Full time

    Company DescriptionBastion data solutions..Role DescriptionThis is a full-time on-site role for a Cloud SRE Architect / DevOps Engineer at Bastion Data Solutions in Chennai. The role involves day-to-day tasks such as Site Reliability Engineering, troubleshooting, software development, system administration, and infrastructure management.QualificationsSite...

  • Bastion Data Solutions | Cloud SRE Architect

    4 weeks ago

    chennai, India Bastion Data Solutions Full time

    Company DescriptionBastion data solutions..Role DescriptionThis is a full-time on-site role for a Cloud SRE Architect / DevOps Engineer at Bastion Data Solutions in Chennai. The role involves day-to-day tasks such as Site Reliability Engineering, troubleshooting, software development, system administration, and infrastructure management.QualificationsSite...

  • Bastion Data Solutions | Cloud SRE Architect

    1 month ago

    chennai, India Bastion Data Solutions Full time

    Company DescriptionBastion data solutions.. Role DescriptionThis is a full-time on-site role for a Cloud SRE Architect / DevOps Engineer at Bastion Data Solutions in Chennai. The role involves day-to-day tasks such as Site Reliability Engineering, troubleshooting, software development, system administration, and infrastructure management. QualificationsSite...

  • Data Management Specialist

    2 weeks ago

    Chennai, Tamil Nadu, India Wavicle Data Solutions Full time

    Job Title: Data Management Specialist**Overview**We are seeking an experienced Data Management Specialist to join our team at Wavicle Data Solutions. The successful candidate will play a pivotal role in architecting and implementing data solutions, leveraging their expertise in managing data-related projects, including Data Engineering and Data Analytics...

  • Senior Data Architect

    2 weeks ago

    Chennai, Tamil Nadu, India Wavicle Data Solutions Full time

    Job Title: Senior Data Architect**Overview**We are seeking an experienced Senior Data Architect to join our team at Wavicle Data Solutions. The successful candidate will play a pivotal role in architecting and implementing data solutions, leveraging their expertise in managing data-related projects, including Data Engineering and Data Analytics...

  • Data Solutions Architect Lead

    3 days ago

    Chennai, Tamil Nadu, India Wavicle Data Solutions Full time

    Wavicle Data Solutions is seeking a highly experienced Data Solutions Architect Lead to join our team.Estimated salary for this role in the United States: $160,000 - $240,000 per year.About the RoleWe are looking for a seasoned professional with a strong background in data engineering and analytics. The ideal candidate will have experience leading...

  • Wavicle Data Solutions | Technical Project Manager | chennai

    1 month ago

    chennai, India Wavicle Data Solutions Full time

    Technical Project Manager: Job Summary: We are seeking an experienced Technical Project Manager with a strong background in managing data-related projects, including Data Engineering and Data Analytics initiatives. The ideal candidate will play a pivotal role in architecting and implementing data solutions and have a proven track record in Agile...

  • Wavicle Data Solutions | Technical Project Manager | chennai

    1 month ago

    chennai, India Wavicle Data Solutions Full time

    Technical Project Manager: Job Summary: We are seeking an experienced Technical Project Manager with a strong background in managing data-related projects, including Data Engineering and Data Analytics initiatives. The ideal candidate will play a pivotal role in architecting and implementing data solutions and have a proven track record in Agile...

  • NTT DATA Business Solutions | Cloud Engineer

    3 weeks ago

    Chennai, India NTT DATA Business Solutions Full time

    NTT DATA Business SolutionsWe want to shape the future with vision and innovation.Be part of it and develop your full potential!As part of the global NTT DATA Group, one of the most successful IT service providers in the world, we specialize in value-added SAP solutions as NTT DATA Business Solutions. With over 10,000 employees in more than 30 countries, we...

  • Crayon Data | AI Architect

    4 days ago

    chennai, India Crayon Data Full time

    Role:AI Architect Exp: 10 to 15 yrs Location: Chennai(Hybrid) In non-cloud Large Language Model (LLM) deployments, the AI Architect serves as a strategic leader, driving the alignment of AI solutions with business objectives while addressing critical challenges like scalability, governance, and security. Their role ensures sustainable and impactful AI...

  • NTT DATA Business Solutions | Cloud Engineer

    3 weeks ago

    chennai, India NTT DATA Business Solutions Full time

    NTT DATA Business Solutions We want to shape the future with vision and innovation. Be part of it and develop your full potential! As part of the global NTT DATA Group, one of the most successful IT service providers in the world, we specialize in value-added SAP solutions as NTT DATA Business Solutions. With over 10,000 employees in more than 30 countries,...

  • Cloud sre architect

    4 weeks ago

    Chennai, India Bastion Data Solutions Full time

    Company DescriptionBastion data solutions..Role DescriptionThis is a full-time on-site role for a Cloud SRE Architect / Dev Ops Engineer at Bastion Data Solutions in Chennai. The role involves day-to-day tasks such as Site Reliability Engineering, troubleshooting, software development, system administration, and infrastructure management.QualificationsSite...

  • NTT DATA Business Solutions | Cloud Engineer

    3 weeks ago

    chennai, India NTT DATA Business Solutions Full time

    NTT DATA Business Solutions We want to shape the future with vision and innovation.Be part of it and develop your full potential!As part of the global NTT DATA Group, one of the most successful IT service providers in the world, we specialize in value-added SAP solutions as NTT DATA Business Solutions. With over 10,000 employees in more than 30 countries, we...

  • Cloud SRE Architect

    1 month ago

    Chennai, India Bastion Data Solutions Full time

    Company Description Bastion data solutions.. Role Description This is a full-time on-site role for a Cloud SRE Architect / DevOps Engineer at Bastion Data Solutions in Chennai. The role involves day-to-day tasks such as Site Reliability Engineering, troubleshooting, software development, system administration, and infrastructure management. ...

  • Cloud SRE Architect

    4 weeks ago

    Chennai, India Bastion Data Solutions Full time

    Company Description Bastion data solutions.. Role Description This is a full-time on-site role for a Cloud SRE Architect / DevOps Engineer at Bastion Data Solutions in Chennai. The role involves day-to-day tasks such as Site Reliability Engineering, troubleshooting, software development, system administration, and infrastructure management. Qualifications...