Lead Data Platform Engineer

Position : Lead Data Platform EngineerLocation : Remote (collaboration across time zones), India or LATAM preferredEngagement Type : ContractLanguage : Fluent EnglishExperience : 9+ years in Data Engineering / Backend Engineering / DevOps, including leadership of a team (or end-to-end ownership of large, multi-stakeholder projects)About the RoleWe’re looking for a hands-on Lead Data Platform Engineer to both build/operate production-grade data pipelines and lead delivery. You’ll set technical direction, run design reviews and code reviews, coach engineers, and coordinate work across time zones. Day to day you’ll work across ingestion, transformation, orchestration, and APIs; automate environments with Terraform; instrument observability; and write tests to keep things reliable. Azure experience is preferred; strong AWS experience is also acceptable. You should communicate clearly (async chat and live sessions), learn fast, and use AI coding tools (e.g., Claude Code) effectively without relying on them exclusively.Key ResponsibilitiesTechnical leadership & delivery : Lead a small squad or workstream; scope milestones; manage dependencies and risks; unblock engineers; communicate status to stakeholders; land outcomes on timeArchitecture & direction : Define target states and data contracts; make build/buy calls; author RFCs; drive cost/performance guardrails and reliability patternsData pipelines : Design, build, and operate reliable batch and change-driven pipelines; schedule/orchestrate jobs; handle schema evolution and failures gracefullyTransformations & modeling : Implement clean, tested merge/transformation pipelines; produce models that are easy to consume for products and analytics; tune SQL for performanceAPIs & integration : Build or collaborate on APIs for data access/updates; design request/response contracts; handle auth (OAuth2/OIDC/JWT), idempotency, validation, and auditingInfrastructure & DevOps : Provision/manage cloud resources with Terraform; automate build/test/deploy with CI/CD; write solid shell scripts for glue/ops tasksObservability & reliability : Instrument logs/metrics/traces; define SLOs and on-call runbooks; lead incident reviews and drive root-cause preventionSecurity & compliance : Apply least-privilege access, secrets management, and encryption in transit/at rest; partner on compliance requirementsQuality & standards : Champion unit/integration/data-quality tests (including SQL); enforce code health via reviews, linters, and CI checksCollaboration & docs : Operate async-first; produce concise docs (data contracts, mappings, runbooks, RFCs); collaborate closely with product/engineering/analytics across time zonesRequired Skills & QualificationsProven leadership : led engineers or owned large, multi-team projects; comfortable running design reviews and giving actionable feedbackStrong SQL(tuning, indexing, query plans) and solid RDBMS fundamentals (e.g., Postgres, SQL Server, MySQL)Pythonfor data work and services; comfortable with shell scripting (bash preferred; Powershell acceptable)Terraformand IaC fundamentals; CI/CD experience (version control workflows, automated tests, environment promotion)Cloud : Azure preferred (storage, compute, identity, monitoring). If not Azure, very strong AWS backgroundData integration : Ingestion (batch + change-driven), orchestration, schema/versioning, resilient retries/replaysWorkflow orchestration : Prefect preferred (flows/deployments, retries/backoff, scheduling, observability); Airflow/Dagster also acceptableAPIs & auth : REST patterns, pagination, validation, rate limiting, OAuth2/OIDC/JWT.Observability : Logs/metrics/traces and actionable alerts/dashboardsTesting mindset : Habit of writing tests for code and SQL; fixtures/test data; CICommunication : Clear written/verbal comms; practiced cross-timezone collaborationAI tooling : Ready to use Claude Code (and similar assistants) to accelerate work—while exercising judgment and writing your own code/testsPreferred Qualifications (Nice to Have)Experience mentoring, interviewing, onboarding, and raising code quality standardsRegulatory/compliance awareness(e.g., data protection or public-sector standards) and how it impacts design/operationsAnalytics exposure(dim/fact modeling, BI consumption patterns)Data platform architectureexposure (storage/layout choices, catalog/lineage, governance concepts)Familiarity with Kafka, streaming/message queues, and columnar formats (e.g., Parquet)Experience with cloud monitoring stacks (Azure Monitor/Application Insights, CloudWatch, Datadog, New Relic, etc.)Engagement DetailsDuration : 10+ months (extension/full-time possible)Schedule : Flexible, with overlap for key meetings across time zonesCompensation : Competitive, based on experienceMode : Remote; available for virtual collaboration and on-call windows as agreedHow to ApplyPlease share:Resume/CVhighlighting data engineering, DevOps/IaC, and/or API/auth workBrief note(one paragraph) about a recent pipeline/service: requirements, your role, tech, validation/tests, deployment/ops, outcomeLinksto code samples or design docs (if available/shareable)Apply via LinkedIn or email