Principal Data Engineer

1 day ago

bangalore district, India Autodesk Full time

#25WD9141 Position Overview We are seeking a highly experienced Principal Engineer to lead the design, development, and evolution of our Batch Processing platform , which powers Autodesk’s Analytics Data Platform (ADP). This role requires deep technical expertise in distributed data systems, large-scale pipeline orchestration, and hands-on leadership in shaping next-generation data platform capabilities. You will partner closely with Engineering Managers, Architects, Product teams, and Partner Engineering to modernize our data lakehouse architecture, deliver highly reliable data pipelines, and establish technical excellence across ingestion, processing, and governance. Responsibilities Technical Leadership Define and drive the technical strategy for Batch Processing within ADP Design and scale PySpark based distributed processing frameworks integrated with Airflow for orchestration Champion Apache Iceberg-based data lakehouse adoption (versioning, branching, WAP, indexing) Introduce performance optimization patterns for data pipelines (e.g., partition pruning, caching, resource tuning) Establish patterns for metadata-driven, intent-based data processing aligned with AI-assisted pipelines Architecture & Delivery Partner with architects to design multi-tenant, secure, and compliant (SOC2/GDPR/CCPA) batch processing services Define reference architectures and reusable frameworks for cross-domain processing workloads Lead technical reviews, solution designs, and architectural forums across ADP Guide the evolution of Airflow APIs and SDKs to enable scalable pipeline self-service Mentorship & Collaboration Mentor senior engineers, staff engineers, and tech leads within the Batch Processing and adjacent teams Partner with the Batch Ingestion and Data Security teams to deliver unified ingestion + processing flows Innovation & Impact Drive modernization initiatives to migrate away from legacy tools Pioneer AI-augmented data engineering practices (e.g., pipeline optimization agents, anomaly detection) Ensure scalability, cost-efficiency, and reliability for thousands of production pipelines across Autodesk Influence company-wide data engineering strategy by contributing thought leadership and whitepapers Minimum Qualifications 8+ years of experience in software/data engineering, with at least 3 years in big data platform leadership roles Expert in distributed data processing (Spark, PySpark, Ray, Flink) Deep experience with workflow orchestration (Airflow, Dagster, Prefect) Strong hands-on expertise in Lakehouse technologies (Iceberg, Delta, Hudi) and cloud platforms (AWS/Azure/GCP) Proven track record in architecting secure, multi-tenant, and compliant data platforms Skilled in SQL/NoSQL databases, Snowflake, Glue, and modern metadata/catalog tools Strong problem-solving, communication, and cross-geo collaboration skills Experience mentoring engineers, building strong technical culture, and influencing at scale Preferred Qualifications Exposure to AI/ML-driven data engineering (optimizers, anomaly detection, auto-scaling) Experience with data governance, lineage, and observability tools (Atlan, Databand, Collibra, etc.) Familiarity with streaming + batch hybrid architectures (Kappa/Lambda)#LI-KS2

  • Principal Engineer

    1 day ago

    bangalore district, India Coffeee.io Full time

    Role- principal Engineer Location- Bangalore (Hybrid) Experience- 10-12 years Employment- Full time About the Role We are looking for a highly skilled Principal Engineer with strong technical expertise and proven experience in engineering leadership. This role is ideal for someone who enjoys being hands-on with coding while also managing and mentoring a...

  • Principal Engineer

    2 weeks ago

    bangalore district, India Recro Full time

    Designation - Principal Engineer Experience - 13+ Years Skills - Java, React, SQL, AWS Location - Bangalore ( Marathalli - Hybrid) Required Skills and Abilities: Expertise in Java (J2EE, Spring Boot, Microservices) for enterprise application development and maintenance. Strong proficiency in React for front-end development and integration. Excellent...

  • Principal Data Engineer

    3 days ago

    bangalore, India Autodesk Full time

    #25WD9141Position Overview We are seeking a highly experienced Principal Engineer to lead the design, development, and evolution of our Batch Processing platform, which powers Autodesk’s Analytics Data Platform (ADP). This role requires deep technical expertise in distributed data systems, large-scale pipeline orchestration, and hands-on leadership in...

  • Principal Database Engineer

    24 hours ago

    bangalore district, India Talentiser Full time

    Principal Database Engineer, Bangalore Position Overview We are seeking an experienced Principal Database Engineer to lead our database infrastructure initiatives with a primary focus on Microsoft SQL Server environments. This senior-level position requires deep technical expertise, strategic thinking, and leadership capabilities to drive database...

  • Principal Engineer – Cloud

    5 days ago

    bangalore, India Bean HR Consulting Full time

    Job Title: Principal Engineer – Cloud & DataLocation: Bangalore (BLR) Experience: 12–16 yearsRole Overview:Lead the design and development of cloud-based applications. Architect scalable, secure, and high-performance systems, mentor the engineering team, and collaborate with product and architecture teams to deliver high-quality solutions.Key...

  • Principal Data Engineer

    2 days ago

    Bangalore, India Autodesk Full time

    #25WD9141 Position Overview We are seeking a highly experienced Principal Engineer to lead the design, development, and evolution of our Batch Processing platform , which powers Autodesk’s Analytics Data Platform (ADP). This role requires deep technical expertise in distributed data systems, large-scale pipeline orchestration, and hands-on leadership in...

  • Principal Engineer

    23 hours ago

    bangalore district, India Razorpay Full time

    A Principal engineer at Razorpay is well-grounded - innovative, quality focussed, product thinker, business conscious - not mere coders, of course, with deeper technical context and experience to build quality software. You should be able to see nuances and elements that others might not. Engineering creates a significant impact across different areas,...

  • Principal Data Engineer

    2 days ago

    Bangalore, India Autodesk Full time

    We are seeking a highly experienced Principal Engineer to lead the design, development, and evolution of our Batch Processing platform , which powers Autodesk’s Analytics Data Platform (ADP). This role requires deep technical expertise in distributed data systems, large-scale pipeline orchestration, and hands-on leadership in shaping next-generation data...

  • Principal Engineer

    24 hours ago

    bangalore, India Coffeee.io Full time

    Role- principal EngineerLocation- Bangalore (Hybrid)Experience- 10-12 yearsEmployment- Full timeAbout the RoleWe are looking for a highly skilled Principal Engineer with strong technical expertise andproven experience in engineering leadership. This role is ideal for someone who enjoys beinghands-on with coding while also managing and mentoring a small team....

  • Data Engineer

    2 weeks ago

    bangalore, India Response Informatics Full time

    🚀 We’re Hiring | Data Engineering Experts (Hyderabad / Remote)We’re looking for passionate and experienced Data Engineering professionals to join our growing team.If you love building scalable data pipelines, optimizing cloud-based workflows, and leading innovation in data infrastructure — we’d love to meet you!🔹 Open Positions:Data Engineer...