Senior Data Engineer

Senior Data Engineer Location: Coimbatore Experience: 3–5 Years Work Mode: On-site About the Role We are seeking a Senior Data Engineer who will take end-to-end ownership of designing, building, and scaling data pipelines, integrations, and analytics layers for our Enterprise AI Operational Platform . The role involves integrating data from construction-focused enterprise tools such as Sage 300, Paperless, WorkMax, Procore, Kojo, CRM platforms , and other financial, procurement, and field applications to create a unified operational data ecosystem . You will architect scalable systems, build automated ETL/ELT workflows, standardize data governance, and enable real-time insights, predictive analytics, and automated operational intelligence across the organization. This position suits someone who brings strong technical depth, system architecture capability, and hands-on experience building large-scale data systems using Microsoft Azure and/or Snowflake . Key Responsibilities Data Architecture & System Design Architect modular, scalable, and secure data platforms. Design centralized data models to support analytics, forecasting, AI, and automation workloads . Define data governance standards including lineage, quality, and retention. Produce HLD/LLD architecture diagrams, integration maps, and technical documentation. Build frameworks that support complex cross-system workflows. Integrations Across Enterprise Tools (Sage 300, Paperless, WorkMax, Procore, Kojo, CRM, etc.) Lead integrations across construction and operations software including: Sage 300 (Financials, Job Costing, AP/AR, GL) Paperless (Document workflows, invoice automation) WorkMax (Timecards, units, production, field data) Procore (Project management, RFIs, submittals, schedules) Kojo (Procurement, materials management) CRM systems (Sales pipeline, customer records) Develop connectors using REST APIs, GraphQL, Webhooks, Secure File Transfer, and Database links . Standardize schemas across tools to create a single source of truth . Ensure secure and reliable data ingestion from on-prem and cloud systems. Implement robust incremental sync, batch loads, and real-time event-based ingestion. ETL/ELT Development & Data Pipelines Build automated ETL/ELT workflows using Airflow, NiFi, Azure Data Factory, or custom frameworks . Ensure pipelines are fault-tolerant, versioned, and optimized for performance . Handle complex transformations, reconciliations, and multi-system data validation. Develop end-to-end data ingestion frameworks supporting large structured and unstructured datasets. Deploy pipelines into production with CI/CD practices. Data Warehousing, Modeling & Real-Time Reporting Build enterprise-wide data warehouse layers using Snowflake (preferred) , Azure SQL, or cloud data stores. Maintain real-time operational data models for job cost, purchasing, productivity, CRM, safety, compliance, and field activity. Create unified analytical datasets for dashboards and business intelligence. Support executive and departmental reporting by eliminating manual spreadsheets and fragmented data silos. Ensure high data accuracy, reliability, and performance at scale. Predictive Analytics & AI Enablement Prepare AI-ready datasets for forecasting, anomaly detection, productivity modeling, and operational risk scoring. Collaborate with data scientists/AI engineers to build and deploy ML pipelines. Develop streaming/near-real-time data flows to support AI-driven decision automation. Integrate predictive insights back into operation systems (alerts, workflows, dashboards). Automation & Operational Workflows Automate KPI tracking, threshold-based alerts, and data validation rules. Build automated workflows for: RFIs CORs Contracts Vendor documents Field reporting Implement monitoring frameworks for pipeline health, error handling, retry logic, and SLA compliance. Documentation & Knowledge Sharing Create detailed documentation for data models, transformations, integrations, and pipelines. Conduct internal training and handover sessions to engineering, analytics, and operations teams. Maintain continuous improvement of data standards and reusable components. Skills & Experience Required Core Experience 3–5 years of hands-on Data Engineering / Data Platform development . Strong expertise in API-based integrations , especially with enterprise applications. Practical experience with: Python , FastAPI, Node.js ETL frameworks: Airflow / NiFi / Azure Data Factory Snowflake (strongly preferred) or Azure-based data warehousing SQL + cloud databases Azure Functions, Storage, Data Lake environments Data Architecture & Modeling Strong data modeling principles (Kimball, Data Vault, or custom patterns). Building scalable data lakes, warehouse schemas, and analytical data marts. Tools & Platforms Dashboarding tools: Power BI / Tableau Experience working with large structured, semi-structured, and unstructured datasets . Background in financial or field operations workflows (preferred). Preferred Knowledge Working with industry tools such as: Sage 300 Procore WorkMax Kojo Paperless Experience in building ML/AI data pipelines. Exposure to Azure DevOps, CI/CD , and cloud deployment best practices. Key Competencies Strong system design and architecture thinking. High ownership, accountability, and delivery focus. Excellent analytical and problem-solving abilities. Clear communication and documentation skills. Ability to work with leadership, operations, finance, field teams, and engineering. Commitment to data accuracy, reliability, and operational performance. What We Offer Opportunity to build a next-generation Enterprise AI Data Platform . Career growth into Data Platform Lead / Engineering Manager roles . Medical insurance and employee benefits. Supportive leadership, high visibility, and impactful responsibilities. Competitive compensation aligned with expertise and contribution.