Principal Data Engineer

4 days ago

New Delhi, India Autodesk Full time

#25WD9141Position OverviewWe are seeking a highly experienced Principal Engineer to lead the design, development, and evolution of our Batch Processing platform, which powers Autodesk’s Analytics Data Platform (ADP). This role requires deep technical expertise in distributed data systems, large-scale pipeline orchestration, and hands-on leadership in shaping next-generation data platform capabilities. You will partner closely with Engineering Managers, Architects, Product teams, and Partner Engineering to modernize our data lakehouse architecture, deliver highly reliable data pipelines, and establish technical excellence across ingestion, processing, and governance.ResponsibilitiesTechnical Leadership- Define and drive the technical strategy for Batch Processing within ADP - Design and scale PySpark based distributed processing frameworks integrated with Airflow for orchestration - Champion Apache Iceberg-based data lakehouse adoption (versioning, branching, WAP, indexing) - Introduce performance optimization patterns for data pipelines (e.g., partition pruning, caching, resource tuning) - Establish patterns for metadata-driven, intent-based data processing aligned with AI-assisted pipelinesArchitecture & Delivery- Partner with architects to design multi-tenant, secure, and compliant (SOC2/GDPR/CCPA) batch processing services - Define reference architectures and reusable frameworks for cross-domain processing workloads - Lead technical reviews, solution designs, and architectural forums across ADP - Guide the evolution of Airflow APIs and SDKs to enable scalable pipeline self-serviceMentorship & Collaboration- Mentor senior engineers, staff engineers, and tech leads within the Batch Processing and adjacent teams - Partner with the Batch Ingestion and Data Security teams to deliver unified ingestion + processing flowsInnovation & Impact- Drive modernization initiatives to migrate away from legacy tools - Pioneer AI-augmented data engineering practices (e.g., pipeline optimization agents, anomaly detection) - Ensure scalability, cost-efficiency, and reliability for thousands of production pipelines across Autodesk - Influence company-wide data engineering strategy by contributing thought leadership and whitepapersMinimum Qualifications- 8+ years of experience in software/data engineering, with at least 3 years in big data platform leadership roles - Expert in distributed data processing (Spark, PySpark, Ray, Flink) - Deep experience with workflow orchestration (Airflow, Dagster, Prefect) - Strong hands-on expertise in Lakehouse technologies (Iceberg, Delta, Hudi) and cloud platforms (AWS/Azure/GCP) - Proven track record in architecting secure, multi-tenant, and compliant data platforms - Skilled in SQL/NoSQL databases, Snowflake, Glue, and modern metadata/catalog tools - Strong problem-solving, communication, and cross-geo collaboration skills - Experience mentoring engineers, building strong technical culture, and influencing at scalePreferred Qualifications- Exposure to AI/ML-driven data engineering (optimizers, anomaly detection, auto-scaling) - Experience with data governance, lineage, and observability tools (Atlan, Databand, Collibra, etc.) - Familiarity with streaming + batch hybrid architectures (Kappa/Lambda)#LI-KS2

  • Principal Data Engineer

    3 weeks ago

    New Delhi, India Xebia Full time

    We’re Hiring: Principal Data Engineer | Any Xebia Location (Hybrid, 3 days in office per week)Location: Any Xebia Location (Hybrid – 3 days in office per week)Experience Level: 10+ years of Data Engineering with 4+ years team leadership experienceStart Date: Immediate to end of SeptemberAbout the Team:As a Principal Data Engineer in our Data Warehouse...

  • Principal Data scientist

    2 weeks ago

    New Delhi, India YASH Technologies Full time

    We are seeking a highly experienced and visionary Senior Principal Data Scientist to join our dynamic team. This role is pivotal in driving our organization's advanced analytics initiatives and shaping our data-driven strategy. As a thought leader, you will lead complex data science projects, mentor junior scientists, and collaborate with cross-functional...

  • Principal LLM Engineer

    2 weeks ago

    New Delhi, India Rapid7 Full time

    Principal LLM EngineerJoin Rapid7: Secure the Future with AIAre you ready to lead the charge in integrating cutting-edge Large Language Models (LLMs) into world-class Cyber Security products?Rapid7 is looking for a Principal LLM Engineer with a rare combination of deep Data Science expertise, mastery of production MLOps, and 13+ years of experience. You...

  • Principal

    2 weeks ago

    New Delhi, India Gyan Ganga Group of Institutions Full time

    Company DescriptionGyan Ganga Institute of Technology and Sciences (GGITS) is a higher education college in Jabalpur, approved by AICTE and affiliated with Rajiv Gandhi Proudyogiki Vishwavidyalaya (University of Technology of M.P.), Bhopal. GGITS offers a range of undergraduate and postgraduate programs such as B.Tech, B.Pharmacy, M.Tech, MBA, and MCA. The...

  • Principal

    2 weeks ago

    New Delhi, India Gyan Ganga Group of Institutions Full time

    Company DescriptionGyan Ganga Institute of Technology and Sciences (GGITS) is a higher education college in Jabalpur, approved by AICTE and affiliated with Rajiv Gandhi Proudyogiki Vishwavidyalaya (University of Technology of M.P.), Bhopal. GGITS offers a range of undergraduate and postgraduate programs such as B.Tech, B.Pharmacy, M.Tech, MBA, and MCA. The...

  • Principal data engineer

    3 weeks ago

    Delhi, India Xebia Full time

    We’re Hiring: Principal Data Engineer | Any Xebia Location (Hybrid, 3 days in office per week)Location:Any Xebia Location (Hybrid – 3 days in office per week)Experience Level:10+ years of Data Engineering with 4+ years team leadership experienceStart Date:Immediate to end of SeptemberAbout the Team:As aPrincipal Data Engineerin ourData Warehouse team ,...

  • Principal Engineer

    2 weeks ago

    New Delhi, India Coffeee.io Full time

    Role- principal Engineer Location- Bangalore (Hybrid) Experience- 10-12 years Employment- Full timeAbout the Role We are looking for a highly skilled Principal Engineer with strong technical expertise and proven experience in engineering leadership. This role is ideal for someone who enjoys being hands-on with coding while also managing and mentoring a small...

  • Principal Engineer

    2 weeks ago

    New Delhi, India CoffeeBeans Full time

    About Us CoffeeBeans Consulting is a tech consulting firm focused on making organizationsAI-readyby structuring their data efficiently across various sources and enabling AI-driven solutions. We specialize indata architecture, pipelines, governance, MLOps, and Gen AIsolutions, helping clients achievefaster go-to-market and cost efficiency . We are looking...

  • Principal Machine Learning Engineer

    1 week ago

    New Delhi, India Informatica Full time

    Principal Machine Learning EngineerInformatica is on a journey to leverage generative AI to simplify cloud data management. Principal Machine Learning Engineer will be responsible for driving the overall architecture and pipelines to enable Machine Learning engineers to build, train and deploy Models at scale across multiple cloud services providers in a...

  • Principal Engineer — Marketing

    6 days ago

    New Delhi, India Wenger & Watson Full time

    Title: Principal Engineer — Marketing & CRMLocation: BangaloreReports to: VP of EngineeringAbout the Role:We’re seeking a Principal Engineer to drive the evolution of our cloud-native, SaaS-based marketing and CRM platform serving banks and credit unions across the U.S. The role will focus on technical vision, scalability, re-architecture, and...