Data Engineer
22 hours ago
Responsibilities and accountabilities:
- Data Collection and Integration
- Collect data from various sources, including databases, APIs, external data providers, NoSQL, unstructured data and streaming sources. Design and implement efficient data pipelines to ensure a smooth flow of information into the data warehouse or storage system.
- Data Storage and Management
- Choose appropriate database systems, optimize data schemas, and ensure data quality and integrity. Consider scalability and performance to handle large volumes of data.
- ETL (Extract, Transform, Load) Processes
- Design ETL pipelines to transform raw data into a format suitable for analysis. This involves data cleansing, aggregation, and enrichment, ensuring the data is usable.
- Big Data Technologies and Cloud Computing
- Big data and OLAP technologies such as Google Cloud Big Query, Hadoop and Spark to efficiently process and analyze massive datasets.
- Enterprise Data Governance
- Define and ensure adherence to the Enterprise Data Governance framework for data practices, policies, and standards both at a technical and a functional level. Work closely with various data owners to develop and implement Data Governance Framework.
- Distributed Systems
- Handle huge data volumes and ensure fault tolerance. Understanding how distributed systems work is essential for data engineers.
- Streaming Data (CDC)
- Work with streaming technologies like Qlik Replicate, Informatica CDC, Apache Kafka to handle and analyze data as it flows in.
Qualification and experience:
- 15+ years of hands-on experience in the field of Data Integration and Governance; out of which at least 10 years of leading a team of Data Engineers and Data Governance specialists.
- Previously held roles leading Data Engineering teams for dynamic, growing organizations.
- Natural enthusiasm to keep up to date on emerging technologies and methodologies to continually improve and advance data solutions where possible.
- Knowledge and experience in building end-to-end Data Pipelines.
- Knowledge and experience in building Enterprise Data Governance Lifecycle based on the Maturity Model.
- Experience in implementing Data Availability, Data Quality, Data Catalogue, Data Lineage & Data Security initiatives across the Enterprise. Having worked in a regulated industry would be a plus.
- Experience in defining and enforcement of Data Engineering and Governance principles and policies across the Enterprise.
- Knowledge of Relational and NoSQL databases. Hands-on experience in T-SQL / PL-SQL programming.
- Bachelor's degree in IT, Computer Science, or a related field; Master's degree will be a plus.
-
india Data Warehouse Engineer Full timeExperience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.
-
Data Warehouse Engineer
1 week ago
India Data Warehouse Engineer Full timeExperience : 2- 5 Years Primary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills. Good to have skills: Power BI, Databricks, Python.
-
Data Warehouse Expert for Analytics Projects
1 week ago
India Data Warehouse Engineer Full timeAbout the RoleWe are seeking an experienced Data Warehouse Engineer to join our team. In this role, you will design, develop, and maintain data warehouses that support business intelligence and analytics initiatives.
-
india Senior Data Integration Engineer Full timeMust Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...
-
india Senior Data Integration Engineer Full timeMust Have Skills/Skill Requirement: Design and architect integration solutions to connect various enterprise applications, systems, and databases. Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications. Utilize Azure Integration Services such as Azure Logic Apps,...
-
Senior Data Integration Engineer
2 weeks ago
India Senior Data Integration Engineer Full timeMust Have Skills/Skill Requirement: Design and architect integration solutions to connect various enterprise applications, systems, and databases. Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications. Utilize Azure Integration Services such as Azure Logic...
-
Cloud Data Engineering Lead
7 days ago
India ITI Data Full timeITI Data is seeking a highly skilled Cloud Data Engineering Lead to spearhead the design and development of scalable data pipelines on AWS. With a strong background in PySpark, Python, and AWS services, you will be responsible for building robust data integration solutions that enable data-driven insights across various internal and external sources.This is...
-
ITI Data | PySpark Data Engineer
1 hour ago
india ITI Data Full timeLocation : IndiaType : Full-timeExperience : 10 – 13 years Functions : Consulting, Finance, Information Technology, Big Data Engineering Industries : Capital Markets, Investment Banking, Alternative Investments, Financial Services, Management Consulting, Information Technology and Services, HealthcareJob DescriptionWe are looking for a PySpark solutions...
-
Lead data engineer
3 weeks ago
India Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, Py Spark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...
-
Lead Data Engineer
4 weeks ago
India Wavicle Data Solutions Full timeJob Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...
-
Lead Data Engineer
3 weeks ago
India Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...
-
Aws data engineer
2 weeks ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on Py Spark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
Data Platform Engineer
3 days ago
India OSD Data Services Full timeData Platform EngineerLocation: RemoteType: InternshipAbout UsAt OSD Data, we’re redefining how businesses leverage their data. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation. With a focus on data lakehouse technologies, we’ve built a...
-
Data Platform Engineer
2 days ago
India OSD Data Services Full timeData Platform Engineer Location : Remote Type : Internship About Us At OSD Data, we’re redefining how businesses leverage their data. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation. With a focus on data lakehouse technologies, we’ve...
-
AWS Data Engineer
3 months ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
1 month ago
india ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
4 months ago
India ITI Data Full timeJob DescriptionWe are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There...
-
AWS Data Engineer
4 weeks ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
4 months ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
Lead Data Engineer: Strategic Data Architect
4 weeks ago
India Wavicle Data Solutions Full timeWavicle Data Solutions is seeking an experienced Lead Data Engineer to play a pivotal role in architecting and implementing data solutions.About the RoleThis position offers a competitive salary of $180,000 - $220,000 per annum, commensurate with experience. Key ResponsibilitiesDesign and develop scalable and efficient data pipelines using Python and...