Data Engineer

About the role: Role Overview: The Data Engineer is responsible for building and maintaining data infrastructure that enables organizations to store, process, and analyze large volumes of data. This role focuses on developing robust batch and streaming data pipelines to create data products, derive insights, and support analytics initiatives. The engineer plays a crucial part in ensuring data systems are reliable, efficient, and scalable, which in turn supports the business's ability to make data -driven decisions. This is a client -facing role, requiring strong communication skills to interact with clients regularly and translate business requirements into effective technical solutions.Requirements Key Responsibilities: Design and Develop Data Pipelines: Create, test, and maintain end -to -end data pipelines for batch processing as well as real -time streaming and event -driven data flows. Ensure these pipelines are robust and efficient. Enabling seamless integration of data from multiple sources into centralized systems, wherever required.Ensure Data Quality and Reliability: Implement data quality checks, validation rules, and monitoring on pipelines to guarantee high integrity of data. Automate workflows where possible to enhance reliability, and promptly troubleshoot any issues to minimize downtime.Ensure Data Quality and Reliability: Implement data quality checks, validation rules, and monitoring on pipelines to guarantee high integrity of data. Automate workflows where possible to enhance reliability, and promptly troubleshoot any issues to minimize downtime.Collaborate with Stakeholders: Work closely with data analysts, data scientists, and other engineering team members to understand data needs and ensure the platform meets business requirements. Liaise with client teams regularly to gather requirements and present data solutions that address their problems.Data Infrastructure Management: Support the underlying data infrastructure (e.g., databases, cloud storage, processing frameworks) to ensure high availability and optimal performance of data systems. Uphold best practices in data security and governance to protect sensitive information and comply with relevant policies.Required Skills and Qualifications:Strong SQL and Python skills: Proficiency in SQL for data querying and transformation, and expertise in Python for scripting and automation of data tasks. Ability to write clean, efficient code to handle data processing at scale.Experience with Data Pipelines: Hands -on experience building or maintaining ETL/ELT processes and data pipelines. Familiarity with both batch processing and streaming data architectures (e.g., using messaging or streaming platforms) is highly desirable.API Integration: Experience working with APIs to fetch or send data. Comfort in integrating third -party data sources or services via RESTful APIs, including authentication and data format handling.Knowledge of Data Systems: Familiarity with data storage and processing technologies such as relational databases, data warehouses, or big data tools. Exposure to cloud data platforms (AWS, Azure, GCP) and modern data integration tools is a plus.Problem -Solving and Communication: Strong analytical thinking to solve data -related problems. Excellent communication skills with the ability to explain technical concepts to non -technical stakeholders and clients. Experience in a client -facing or consulting environment is beneficial, as this role involves regular client interaction, is a plus.Education: Bachelor's degree in Computer Science, Engineering, or a related field (or equivalent practical experience). A solid foundation in data structures, algorithms, and software engineering principles is expected.