Data Integration Engineer

Description :Position : Data Integration EngineerExperience : 2-5 YearsLocation : BangaloreIndustry Type : Logistics / Supply Chain Technology (Assumed)Education : Bachelor's degree in Computer Science or a related field.Job Summary :We are seeking a proactive Data Integration Engineer with 2-5 years of hands-on experience in building and optimizing robust data pipelines. The role requires strong proficiency in Python (including Pandas/NumPy), an excellent grasp of data structures and algorithms, and practical experience with ETL/data integration workflows. The Engineer will be responsible for designing and managing data exchange across various formats (JSON/XML, CSV, EDI) and sources (REST APIs, relational/NoSQL databases), with mandatory experience in workflow orchestration using Apache Airflow.Job Description :Data Pipeline Design and Development (ETL) :- Design, build, and maintain robust ETL/data pipelines or data integration workflows to ensure the seamless, high-volume flow of data between internal and external systems.- Demonstrate mandatory Proficiency in Python, utilizing related libraries such as Pandas and NumPy for complex data manipulation, cleansing, and transformation tasks.- Apply a strong understanding of object-oriented programming (OOP) principles, data structures, and algorithms to develop scalable and efficient code.- Ensure the quality and reliability of integrated data by implementing rigorous testing and error handling within all ETL processes.Data Management and Integration Technologies :- Possess comprehensive knowledge of both relational and non-relational databases, including expertise in data modeling and normalization techniques to optimize storage and retrieval efficiency.- Work hands-on with various data formats, including structured files like JSON and XML, and flat file formats such as CSV and EDI (Electronic Data Interchange), which are critical for the logistics domain.- Develop and manage data retrieval mechanisms by working extensively with REST APIs for integrating external partner and client data sources.- Create, schedule, and manage complex workflows using Apache Airflow to orchestrate pipelines, manage task dependencies, and monitor job execution effectively.Collaboration and System Optimization :- Analyze data flow requirements and perform detailed impact analysis for changes to upstream and downstream systems.- Demonstrate the ability to collaborate effectively within teams and exhibit the discipline to work independently when necessary to meet project deadlines.- Contribute to system optimization efforts, focusing on improving the performance, resilience, and cost-effectiveness of data integration infrastructure.Required Skills & Qualifications :- Experience : Mandatory 2-5 years of experience in data engineering or integration.- Core Programming : Proficiency in Python and related libraries (Pandas and NumPy).- Fundamentals : Strong understanding of object-oriented programming, data structures, and algorithms.- Databases : Knowledge of relational and non-relational databases, data modeling, and normalization.- Pipelines : Hands-on experience designing and building ETL/data pipelines or data integration workflows.- Orchestration : Experience creating workflows using Apache Airflow.- Formats/APIs : Experience with structured (JSON/XML) and flat files (CSV, EDI), and working with REST APIs.- Education : Bachelor's degree in Computer Science or a related field.Preferred Skills :- Cloud : Exposure to AWS or other cloud platforms (e.g., Azure, GCP) and cloud-native data services.- Client Engagement : Prior client-facing experience for requirement gathering and technical discussion.- Domain : Familiarity with data standards and integration challenges within the Logistics or Supply Chain domain. (ref:hirist.tech)