Data Engineer
1 month ago
We are looking for a skilled Data Engineering Analyst with expertise in PySpark, Python, Hadoop, and Azure Databricks to join our team. You will play a key role in data integration, transformation, and analysis, working with large datasets to deliver insights that drive business decisions. In this role, you will collaborate with cross-functional teams, including data scientists, data engineers, and business analysts, to develop and maintain data pipelines, and ensure data quality and consistency across systems.
Key Responsibilities
Design, develop, and optimize ETL pipelines for data ingestion, transformation, and loading, using PySpark and Azure Databricks.
Implement data processing and transformation logic in Python and PySpark to handle large datasets efficiently.
Work with Hadoop-based data lakes and Azure cloud storage solutions to manage and retrieve data as required.
Collaborate with data scientists and analysts to provide data-driven insights and solutions.
Perform data profiling, validation, and quality assurance to ensure data accuracy and consistency.
Optimize data models and pipelines for performance, scalability, and reliability.
Utilize Azure Databricks for data processing workflows, managing resources, and ensuring job efficiency.
Monitor, troubleshoot, and resolve data pipeline issues to ensure minimal downtime and data accuracy.
Contribute to documentation and best practices for data engineering processes and standards.
Required Skills and Qualifications
Education : Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
Experience : 3+ years of experience in data engineering, with hands-on experience in PySpark, Python, Hadoop, and Azure Databricks.
Technical Skills :
Strong experience with Python and PySpark for data transformation and analysis.
Proficiency in Hadoop ecosystems (HDFS, Hive, HBase) for data storage and retrieval.
Experience with Azure Databricks and other Azure data services, including Azure Data Lake, Azure Blob Storage, and Azure Synapse.
Strong SQL skills for data manipulation, querying, and validation.
Knowledge of ETL processes and tools, including performance optimization techniques.
Analytical Skills :
Strong problem-solving skills with the ability to troubleshoot complex data issues.
Proficiency in handling large datasets and performing data profiling and cleansing.
Soft Skills :
Excellent communication and teamwork abilities to collaborate with cross-functional teams.
Strong attention to detail and ability to work independently and manage multiple tasks effectively.
-
tamil nadu, India Wavicle Data Solutions Full timeJob Description:We are seeking a highly skilled and dynamic Senior Engineer in Business Intelligence (BI) to spearhead our BI initiatives. The ideal candidate will have expertise in AWS QuickSight, Tableau and Power BI, along with strong cloud knowledge, communication skills, and a background in ETL/Data Warehousing.Key Responsibilities:Develop and lead the...
-
tamil nadu, India Wavicle Data Solutions Full timeJob Description:We are seeking a highly skilled and dynamic Senior Engineer in Business Intelligence (BI) to spearhead our BI initiatives. The ideal candidate will have expertise in AWS QuickSight, Tableau and Power BI, along with strong cloud knowledge, communication skills, and a background in ETL/Data Warehousing.Key Responsibilities:Develop and lead the...
-
tamil nadu, India Wavicle Data Solutions Full timeAbout the Role: We are seeking a talented and passionate Java Software Engineer to join our growing team. In this role, you will be responsible for designing, developing, and maintaining high-performance Java applications. You will work closely with other engineers and stakeholders to deliver innovative and scalable solutions that meet our business needs....
-
tamil nadu, India Wavicle Data Solutions Full timeAbout the Role:We are seeking a talented and passionate Java Software Engineer to join our growing team.In this role, you will be responsible for designing, developing, and maintaining high-performance Java applications.You will work closely with other engineers and stakeholders to deliver innovative and scalable solutions that meet our business...
-
tamil nadu, India Wavicle Data Solutions Full timeJob Title: Senior Data ScientistWe are seeking a highly skilled Senior Data Scientist with expertise in Generative AI, machine learning, and advanced data science techniques to join our dynamic team. This role involves collaborating with cross-functional teams to develop and deploy scalable AI solutions, driving innovation and business...
-
NTT DATA Business Solutions | Cloud Engineer
2 weeks ago
tamil nadu, India NTT DATA Business Solutions Full timeNTT DATA Business Solutions We want to shape the future with vision and innovation.Be part of it and develop your full potential!As part of the global NTT DATA Group, one of the most successful IT service providers in the world, we specialize in value-added SAP solutions as NTT DATA Business Solutions. With over 10,000 employees in more than 30 countries,...
-
Senior BI Engineer
4 weeks ago
Tamil Nadu, India Wavicle Data Solutions Full timeJob Description: We are seeking a highly skilled and dynamic Senior Engineer in Business Intelligence (BI) to spearhead our BI initiatives. The ideal candidate will have expertise in AWS QuickSight, Tableau and Power BI, along with strong cloud knowledge, communication skills, and a background in ETL/Data Warehousing. Key Responsibilities: Develop and lead...
-
Senior BI Engineer
4 weeks ago
Tamil Nadu, India Wavicle Data Solutions Full timeJob Description:We are seeking a highly skilled and dynamic Senior Engineer in Business Intelligence (BI) to spearhead our BI initiatives. The ideal candidate will have expertise in AWS QuickSight, Tableau and Power BI, along with strong cloud knowledge, communication skills, and a background in ETL/Data Warehousing.Key Responsibilities:Develop and lead the...
-
Data Engineer
2 months ago
tamil nadu, India Tech Mahindra Full timeHi All,Hiring GCP Data Engineer .Experience : 2- 9 yearsJob Location : Pan IndiaPrimary Skill : Big Query, Data flow, Data Composer, SQL, Python, PySpark• Proven experience as a Data Engineer with a focus on GCP services.• Strong proficiency in GCP services such as Dataflow, BigQuery, Data Composer.• Proficiency in SQL and Python for data manipulation...
-
Data Engineer
4 months ago
tamil nadu, India CYGNVS Full timeData EngineerLocation: Chennai (Hybrid)We’re looking for Data Engineers on our Analytics Engineering team - you’ll collaborate with teams across the globe to build pipelines and systems to deliver data insights critical to our customers and internal teams. You’ll be part of a cross functional team that’s responsible for the full product development...
-
Data Engineer
2 months ago
tamil nadu, India Tata Consultancy Services Full timeTCS Hiring !!!! VIRTUAL DRIVE: Azure Data Engineer Developer : 07-November-24NOTE: If the skills/profile matches and interested, please reply to this email by attaching your latest updated CV and with below few details:Name:Contact Number:Email ID:Highest Qualification in: (Eg. B.Tech/B.E./M.Tech/MCA/M.Sc./MS/BCA/B.Sc./Etc.)Current Organization Name:Total...
-
Senior Java Software Engineer
4 weeks ago
Tamil Nadu, India Wavicle Data Solutions Full timeAbout the Role: We are seeking a talented and passionate Java Software Engineer to join our growing team. In this role, you will be responsible for designing, developing, and maintaining high-performance Java applications. You will work closely with other engineers and stakeholders to deliver innovative and scalable solutions that meet our business...
-
Senior Java Software Engineer
4 weeks ago
Tamil Nadu, India Wavicle Data Solutions Full timeAbout the Role: We are seeking a talented and passionate Java Software Engineer to join our growing team. In this role, you will be responsible for designing, developing, and maintaining high-performance Java applications. You will work closely with other engineers and stakeholders to deliver innovative and scalable solutions that meet our business needs....
-
Senior Java Software Engineer
4 weeks ago
Tamil Nadu, India Wavicle Data Solutions Full timeAbout the Role:We are seeking a talented and passionate Java Software Engineer to join our growing team.In this role, you will be responsible for designing, developing, and maintaining high-performance Java applications.You will work closely with other engineers and stakeholders to deliver innovative and scalable solutions that meet our business...
-
Data Engineer
2 months ago
tamil nadu, India Alp Consulting Ltd. Full timeSUMMARY OF RESPONSIBILITIESDesign and implement ETL processes using Azure Data Factory and SSIS to facilitate data extraction, transformation, and loading from various sources.Develop real-time data streaming solutions using Azure Stream Analytics and Azure Event Hubs to support real-time analytics and reporting.Utilize PySpark and Databricks to perform...
-
Data Engineer
7 months ago
tamil nadu, India Blackstraw.ai Full timeData EngineerCompany Name: Blackstraw.aiOffice Location: Chennai (Work from Office)Job Type: Full-timeExperience: 4+ YearsJob Description: We are looking for Data Engineers to join our team. You will use various methods to transform raw data into useful data systems. For example, you’ll create algorithms and conduct statistical analysis. Overall, you’ll...
-
Senior Data Engineer
2 months ago
tamil nadu, India Deloitte Full timeHi, We are looking for Candidates who are having experience in Data Bricks Engineer Along with any cloud experience.Experience Required: 7- 14 YearsLocation: Coimbatore, Tamil Nadu, IndiaResponsibilitiesDesign, develop, and optimize data workflows and notebooks using Databricks to ingest, transform, and load data from various sources into the data lake.Build...
-
Data Engineer
2 months ago
tamil nadu, India HTC Global Services Full timeData EngineerExperience: 3+ yearsLocation: Chennai/Bangalore/HyderabadNotice Period: Immediate to 15 daysJob Description:Degree in Computer Science or equivalent5 or more years of relevant experience in IT industry.3 to 4 years of data warehouse, data lake and business intelligence implementation experienceTechnical Skills (must have these skillsets) :1 - 2...
-
Data Engineer
1 month ago
tamil nadu, India HTC Global Services Full timeData Engineer Experience: 3+ yearsLocation: Chennai/Bangalore/HyderabadNotice Period: Immediate to 15 daysJob Description:Degree in Computer Science or equivalent5 or more years of relevant experience in IT industry.3 to 4 years of data warehouse, data lake and business intelligence implementation experienceTechnical Skills (must have these skillsets) :1...
-
Senior Data Engineer
2 months ago
tamil nadu, India Zocket Full timeAbout ZocketFounded in 2021, Zocket leverages GenAI to enable businesses to effortlessly create and launch ads in seconds. With Zocket, businesses can manage ads across channels like Facebook, Instagram, Google and WhatsApp in a single platform.Zocket's proprietary world-class tech stack is built with cutting edge Machine learning and Artificial...