Lead data platform engineer

Position : Lead Data Platform Engineer Location : Remote (collaboration across time zones), India or LATAM preferred Engagement Type : Contract Language : Fluent English Experience : 9+ years in Data Engineering / Backend Engineering / Dev Ops, including leadership of a team (or end-to-end ownership of large, multi-stakeholder projects) About the Role We’re looking for a hands-on Lead Data Platform Engineer to both build/operate production-grade data pipelines and lead delivery. You’ll set technical direction, run design reviews and code reviews, coach engineers, and coordinate work across time zones. Day to day you’ll work across ingestion, transformation, orchestration, and APIs; automate environments with Terraform; instrument observability; and write tests to keep things reliable. Azure experience is preferred; strong AWS experience is also acceptable. You should communicate clearly (async chat and live sessions), learn fast, and use AI coding tools (e.g., Claude Code) effectively without relying on them exclusively. Key Responsibilities Technical leadership & delivery : Lead a small squad or workstream; scope milestones; manage dependencies and risks; unblock engineers; communicate status to stakeholders; land outcomes on time Architecture & direction : Define target states and data contracts; make build/buy calls; author RFCs; drive cost/performance guardrails and reliability patterns Data pipelines : Design, build, and operate reliable batch and change-driven pipelines; schedule/orchestrate jobs; handle schema evolution and failures gracefully Transformations & modeling : Implement clean, tested merge/transformation pipelines; produce models that are easy to consume for products and analytics; tune SQL for performance APIs & integration : Build or collaborate on APIs for data access/updates; design request/response contracts; handle auth (OAuth2/OIDC/JWT), idempotency, validation, and auditing Infrastructure & Dev Ops : Provision/manage cloud resources with Terraform; automate build/test/deploy with CI/CD; write solid shell scripts for glue/ops tasks Observability & reliability : Instrument logs/metrics/traces; define SLOs and on-call runbooks; lead incident reviews and drive root-cause prevention Security & compliance : Apply least-privilege access, secrets management, and encryption in transit/at rest; partner on compliance requirements Quality & standards : Champion unit/integration/data-quality tests (including SQL); enforce code health via reviews, linters, and CI checks Collaboration & docs : Operate async-first; produce concise docs (data contracts, mappings, runbooks, RFCs); collaborate closely with product/engineering/analytics across time zones Required Skills & Qualifications Proven leadership : led engineers or owned large, multi-team projects; comfortable running design reviews and giving actionable feedback Strong SQL (tuning, indexing, query plans) and solid RDBMS fundamentals (e.g., Postgres, SQL Server, My SQL) Python for data work and services; comfortable with shell scripting (bash preferred; Powershell acceptable) Terraform and Ia C fundamentals; CI/CD experience (version control workflows, automated tests, environment promotion) Cloud : Azure preferred (storage, compute, identity, monitoring). If not Azure, very strong AWS background Data integration : Ingestion (batch + change-driven), orchestration, schema/versioning, resilient retries/replays Workflow orchestration : Prefect preferred (flows/deployments, retries/backoff, scheduling, observability); Airflow/Dagster also acceptable APIs & auth : REST patterns, pagination, validation, rate limiting, OAuth2/OIDC/JWT. Observability : Logs/metrics/traces and actionable alerts/dashboards Testing mindset : Habit of writing tests for code and SQL; fixtures/test data; CI Communication : Clear written/verbal comms; practiced cross-timezone collaboration AI tooling : Ready to use Claude Code (and similar assistants) to accelerate work—while exercising judgment and writing your own code/tests Preferred Qualifications (Nice to Have) Experience mentoring, interviewing, onboarding, and raising code quality standards Regulatory/compliance awareness (e.g., data protection or public-sector standards) and how it impacts design/operations Analytics exposure (dim/fact modeling, BI consumption patterns) Data platform architecture exposure (storage/layout choices, catalog/lineage, governance concepts) Familiarity with Kafka, streaming/message queues, and columnar formats (e.g., Parquet) Experience with cloud monitoring stacks (Azure Monitor/Application Insights, Cloud Watch, Datadog, New Relic, etc.) Engagement Details Duration : 10+ months (extension/full-time possible) Schedule : Flexible, with overlap for key meetings across time zones Compensation : Competitive, based on experience Mode : Remote; available for virtual collaboration and on-call windows as agreed How to Apply Please share: Resume/CV highlighting data engineering, Dev Ops/Ia C, and/or API/auth work Brief note (one paragraph) about a recent pipeline/service: requirements, your role, tech, validation/tests, deployment/ops, outcome Links to code samples or design docs (if available/shareable) Apply via Linked In or email