Senior Data Engineer

Senior Data EngineerLocation: CoimbatoreExperience: 3–5 YearsWork Mode: On-siteAbout the RoleWe are seeking a Senior Data Engineer who will take end-to-end ownership of designing, building, and scaling data pipelines, integrations, and analytics layers for our Enterprise AI Operational Platform.The role involves integrating data from construction-focused enterprise tools such as Sage 300, Paperless, WorkMax, Procore, Kojo, CRM platforms, and other financial, procurement, and field applications to create a unified operational data ecosystem.You will architect scalable systems, build automated ETL/ELT workflows, standardize data governance, and enable real-time insights, predictive analytics, and automated operational intelligence across the organization.This position suits someone who brings strong technical depth, system architecture capability, and hands-on experience building large-scale data systems using Microsoft Azure and/or Snowflake.Key ResponsibilitiesData Architecture & System Design- Architect modular, scalable, and secure data platforms.- Design centralized data models to support analytics, forecasting, AI, and automation workloads.- Define data governance standards including lineage, quality, and retention.- Produce HLD/LLD architecture diagrams, integration maps, and technical documentation.- Build frameworks that support complex cross-system workflows.Integrations Across Enterprise Tools (Sage 300, Paperless, WorkMax, Procore, Kojo, CRM, etc.)- Lead integrations across construction and operations software including:- Sage 300 (Financials, Job Costing, AP/AR, GL)- Paperless (Document workflows, invoice automation)- WorkMax (Timecards, units, production, field data)- Procore (Project management, RFIs, submittals, schedules)- Kojo (Procurement, materials management)- CRM systems (Sales pipeline, customer records)- Develop connectors using REST APIs, GraphQL, Webhooks, Secure File Transfer, and Database links.- Standardize schemas across tools to create a single source of truth.- Ensure secure and reliable data ingestion from on-prem and cloud systems.- Implement robust incremental sync, batch loads, and real-time event-based ingestion.ETL/ELT Development & Data Pipelines- Build automated ETL/ELT workflows using Airflow, NiFi, Azure Data Factory, or custom frameworks.- Ensure pipelines are fault-tolerant, versioned, and optimized for performance.- Handle complex transformations, reconciliations, and multi-system data validation.- Develop end-to-end data ingestion frameworks supporting large structured and unstructured datasets.- Deploy pipelines into production with CI/CD practices.Data Warehousing, Modeling & Real-Time Reporting- Build enterprise-wide data warehouse layers using Snowflake (preferred), Azure SQL, or cloud data stores.- Maintain real-time operational data models for job cost, purchasing, productivity, CRM, safety, compliance, and field activity.- Create unified analytical datasets for dashboards and business intelligence.- Support executive and departmental reporting by eliminating manual spreadsheets and fragmented data silos.- Ensure high data accuracy, reliability, and performance at scale.Predictive Analytics & AI Enablement- Prepare AI-ready datasets for forecasting, anomaly detection, productivity modeling, and operational risk scoring.- Collaborate with data scientists/AI engineers to build and deploy ML pipelines.- Develop streaming/near-real-time data flows to support AI-driven decision automation.- Integrate predictive insights back into operation systems (alerts, workflows, dashboards).Automation & Operational Workflows- Automate KPI tracking, threshold-based alerts, and data validation rules.- Build automated workflows for:- RFIs- CORs- Contracts- Vendor documents- Field reporting- Implement monitoring frameworks for pipeline health, error handling, retry logic, and SLA compliance.Documentation & Knowledge Sharing- Create detailed documentation for data models, transformations, integrations, and pipelines.- Conduct internal training and handover sessions to engineering, analytics, and operations teams.- Maintain continuous improvement of data standards and reusable components.Skills & Experience RequiredCore Experience- 3–5 years of hands-on Data Engineering / Data Platform development.- Strong expertise in API-based integrations, especially with enterprise applications.- Practical experience with:- Python, FastAPI, Node.js- ETL frameworks: Airflow / NiFi / Azure Data Factory- Snowflake (strongly preferred) or Azure-based data warehousing- SQL + cloud databases- Azure Functions, Storage, Data Lake environmentsData Architecture & Modeling- Strong data modeling principles (Kimball, Data Vault, or custom patterns).- Building scalable data lakes, warehouse schemas, and analytical data marts.Tools & Platforms- Dashboarding tools: Power BI / Tableau- Experience working with large structured, semi-structured, and unstructured datasets.- Background in financial or field operations workflows (preferred).Preferred Knowledge- Working with industry tools such as:- Sage 300- Procore- WorkMax- Kojo- Paperless- Experience in building ML/AI data pipelines.- Exposure to Azure DevOps, CI/CD, and cloud deployment best practices.Key Competencies- Strong system design and architecture thinking.- High ownership, accountability, and delivery focus.- Excellent analytical and problem-solving abilities.- Clear communication and documentation skills.- Ability to work with leadership, operations, finance, field teams, and engineering.- Commitment to data accuracy, reliability, and operational performance.What We Offer- Opportunity to build a next-generation Enterprise AI Data Platform.- Career growth into Data Platform Lead / Engineering Manager roles.- Medical insurance and employee benefits.- Supportive leadership, high visibility, and impactful responsibilities.- Competitive compensation aligned with expertise and contribution.