Data Engineer

Position Details:The Data Engineer is responsible for designing, developing, and maintaining robust, scalable, and secure data pipelines and platforms that enable efficient storage, processing, and consumption of data across the enterprise. The role ensures high-quality, timely, and governed data delivery to data scientists, analysts, and business users.The Data Engineer plays a crucial role in building cloud-native data architectures, integrating data from diverse sources, and supporting the full lifecycle of data—from ingestion to transformation to consumption. The role ensures that the data infrastructure is reliable, compliant, and optimized to support advanced analytics and AI use cases across business functions.Qualification: Technical graduate (Engineering degree). Certifications in cloud data platforms (Azure Data Engineer, AWS Big Data, GCP Data Engineer), Big Data tools, or database technologies are preferredExperience: 5–10 years of experience in data engineering or related roles. Proven experience in building and optimizing data pipelines, cloud data architecture, and handling large-scale datasets. Experience working with cross-functional data science, analytics, and IT teams.Key ResponsibilitiesData Pipeline DevelopmentWork Design, develop, and manage scalable ETL/ELT pipelines to extract, transform, and load dataAutomate ingestion from multiple sources including APIs, files, databases, and third-party systemsImplement data ingestion from a variety of sources: ERP, CRM, APIs, file systems, IoT, etc.Implement efficient batch and real-time data processing pipelinesData Integration & Middleware InteractionIntegrate data from source systems via middleware platforms and API-based connectorsLeverage OData, REST, SOAP, and other data protocols to extract data from third-party platforms (e.g., SAP, Salesforce, legacy systems)Collaborate with integration teams to establish secure and reliable connections via middleware (e.g., MuleSoft, Azure Integration Services)Data Modelling & ArchitectureDesign and implement data models for operational and analytical use cases (e.g., star/snowflake schemas) . Leverage knowledge of source systems schemas for ETL design and developmentWork closely with Data Architects and Analysts to ensure alignment with business needsOptimize data storage using partitioning, compression, and indexing strategiesCloud Data Platform EngineeringBuild and manage cloud-native data solutions using services like Azure Data Factory, AWS Glue, or GCP DataflowEnsure optimal performance, scalability, and cost-efficiency of cloud data environmentsSupport migration from legacy systems to modern cloud-based architecturesData Quality & GovernanceImplement data validation, reconciliation, and monitoring frameworksEnsure data accuracy, completeness, and integrity through automated checksCollaborate with data governance teams to ensure metadata management and lineage trackingData Lineage- Document lineage and transformations to support transparency and auditabilityCollaboration and SupportPartner with Data Scientists, Analysts, and BI Developers to deliver data for ML models, dashboards, and reportsProvide support for data issues and troubleshoot pipeline failuresDocument code, workflows, and platform configurations for maintainabilitySecurity & ComplianceEnsure secure access control, encryption, and compliance with data privacy regulationsWork with security teams to perform vulnerability assessments on data systemsImplement logging and monitoring for audit and traceabilityTechnology BackgroundInfrastructure & Systems: Build and Maintain the data pipelines, databases, and other systems that an organization uses to manage its dataData Processing & Transformation: Develop processes to transform raw data into a usable format for analysis and reportingProgramming Languages: Proficiency in languages Python, SQL, Scala, Shell scriptingETL Tools: Experience on ETL tools - Azure Data Factory, Apache NiFi, Talend, InformaticaBig Data Frameworks: Awareness of Apache Spark, Hadoop, KafkaCloud Platforms: Awareness of Azure (preferred), AWS, GCPData Storage: Azure Data Lake, Delta Lake, Snowflake, AWS S3, BigQueryDatabases: MS SQL Server, PostgreSQL, MySQL, MongoDBOrchestration: Airflow, Azure Data Factory, PrefectCI/CD & Version Control: Git, Azure DevOps, JenkinsMonitoring & Logging: Datadog, Prometheus, ELK StackVisualization: Awareness of few visualization tools example- power BI etcData Governance Tools: Responsible for ensuring that all data is stored securely and that investments in security measures are made and regularly maintained example Collibra, Microsoft Purview, Alation (as applicable)About Company:Coromandel International Limited is a leading Indian agrichemical company, part of the Murugappa Group and a subsidiary of EID Parry (owner of approximately 56–63%).Founded in the early 1960s (as Coromandel Fertilisers), the company is currently headquartered in Chennai with its registered office in Hyderabad.They are one of India’s largest private-sector producers of phosphatic fertilizers and the world’s largest manufacturer of neem-based bio-pesticides. Additionally, they lead the market in organic fertilizers and operate the country’s largest agri-retail chain, with 1000+ stores serving over 2 crore farmers.