Principal Data Engineer
2 days ago
#25WD9141 Position Overview We are seeking a highly experienced Principal Engineer to lead the design, development, and evolution of our Batch Processing platform , which powers Autodesk’s Analytics Data Platform (ADP). This role requires deep technical expertise in distributed data systems, large-scale pipeline orchestration, and hands-on leadership in shaping next-generation data platform capabilities. You will partner closely with Engineering Managers, Architects, Product teams, and Partner Engineering to modernize our data lakehouse architecture, deliver highly reliable data pipelines, and establish technical excellence across ingestion, processing, and governance. Responsibilities Technical Leadership Define and drive the technical strategy for Batch Processing within ADP Design and scale PySpark based distributed processing frameworks integrated with Airflow for orchestration Champion Apache Iceberg-based data lakehouse adoption (versioning, branching, WAP, indexing) Introduce performance optimization patterns for data pipelines (e.g., partition pruning, caching, resource tuning) Establish patterns for metadata-driven, intent-based data processing aligned with AI-assisted pipelines Architecture & Delivery Partner with architects to design multi-tenant, secure, and compliant (SOC2/GDPR/CCPA) batch processing services Define reference architectures and reusable frameworks for cross-domain processing workloads Lead technical reviews, solution designs, and architectural forums across ADP Guide the evolution of Airflow APIs and SDKs to enable scalable pipeline self-service Mentorship & Collaboration Mentor senior engineers, staff engineers, and tech leads within the Batch Processing and adjacent teams Partner with the Batch Ingestion and Data Security teams to deliver unified ingestion + processing flows Innovation & Impact Drive modernization initiatives to migrate away from legacy tools Pioneer AI-augmented data engineering practices (e.g., pipeline optimization agents, anomaly detection) Ensure scalability, cost-efficiency, and reliability for thousands of production pipelines across Autodesk Influence company-wide data engineering strategy by contributing thought leadership and whitepapers Minimum Qualifications 8+ years of experience in software/data engineering, with at least 3 years in big data platform leadership roles Expert in distributed data processing (Spark, PySpark, Ray, Flink) Deep experience with workflow orchestration (Airflow, Dagster, Prefect) Strong hands-on expertise in Lakehouse technologies (Iceberg, Delta, Hudi) and cloud platforms (AWS/Azure/GCP) Proven track record in architecting secure, multi-tenant, and compliant data platforms Skilled in SQL/NoSQL databases, Snowflake, Glue, and modern metadata/catalog tools Strong problem-solving, communication, and cross-geo collaboration skills Experience mentoring engineers, building strong technical culture, and influencing at scale Preferred Qualifications Exposure to AI/ML-driven data engineering (optimizers, anomaly detection, auto-scaling) Experience with data governance, lineage, and observability tools (Atlan, Databand, Collibra, etc.) Familiarity with streaming + batch hybrid architectures (Kappa/Lambda)#LI-KS2
-
Principal Engineer
2 days ago
Bangalore Division, India Wenger & Watson Full timeTitle: Principal Engineer — Marketing & CRM Location: Bangalore Reports to: VP of Engineering About the Role: We’re seeking a Principal Engineer to drive the evolution of our cloud-native, SaaS-based marketing and CRM platform serving banks and credit unions across the U.S. The role will focus on technical vision, scalability, re-architecture, and...
-
Principal Data Engineer
7 days ago
bangalore, India Autodesk Full time#25WD9141Position Overview We are seeking a highly experienced Principal Engineer to lead the design, development, and evolution of our Batch Processing platform, which powers Autodesk’s Analytics Data Platform (ADP). This role requires deep technical expertise in distributed data systems, large-scale pipeline orchestration, and hands-on leadership in...
-
Principal Engineer – Cloud
1 week ago
bangalore, India Bean HR Consulting Full timeJob Title: Principal Engineer – Cloud & DataLocation: Bangalore (BLR) Experience: 12–16 yearsRole Overview:Lead the design and development of cloud-based applications. Architect scalable, secure, and high-performance systems, mentor the engineering team, and collaborate with product and architecture teams to deliver high-quality solutions.Key...
-
Principal Data Engineer
6 days ago
Bangalore, India Autodesk Full time#25WD9141 Position Overview We are seeking a highly experienced Principal Engineer to lead the design, development, and evolution of our Batch Processing platform , which powers Autodesk’s Analytics Data Platform (ADP). This role requires deep technical expertise in distributed data systems, large-scale pipeline orchestration, and hands-on leadership in...
-
Principal Data Engineer
6 days ago
Bangalore, India Autodesk Full timeWe are seeking a highly experienced Principal Engineer to lead the design, development, and evolution of our Batch Processing platform , which powers Autodesk’s Analytics Data Platform (ADP). This role requires deep technical expertise in distributed data systems, large-scale pipeline orchestration, and hands-on leadership in shaping next-generation data...
-
Principal Data Engineer
6 days ago
bangalore district, India Autodesk Full time#25WD9141 Position Overview We are seeking a highly experienced Principal Engineer to lead the design, development, and evolution of our Batch Processing platform , which powers Autodesk’s Analytics Data Platform (ADP). This role requires deep technical expertise in distributed data systems, large-scale pipeline orchestration, and hands-on leadership in...
-
Principal Data Engineer
3 days ago
Bangalore - Manyata Tech Park Road, India Commonwealth Bank of Australia Full time ₹ 12,00,000 - ₹ 24,00,000 per yearPrincipal Data Engineer Organization: At CommBank, we never lose sight of the role we play in other people's financial wellbeing. Our focus is to help people and businesses move forward to progress. To make the right financial decisions and achieve their dreams, targets, and aspirations. Regardless of where you work within our organisation, your...
-
Principal Data Engineer
4 days ago
Bangalore - Manyata Tech Park Road, India Commonwealth Bank Full time ₹ 12,00,000 - ₹ 24,00,000 per yearOrganization: At CommBank, we never lose sight of the role we play in other people's financial wellbeing. Our focus is to help people and businesses move forward to progress. To make the right financial decisions and achieve their dreams, targets, and aspirations. Regardless of where you work within our organisation, your initiative, talent, ideas, and...
-
Principal Engineer
2 days ago
bangalore district, India Wenger & Watson Full timeTitle: Principal Engineer — Marketing & CRM Location: Bangalore Reports to: VP of Engineering About the Role: We’re seeking a Principal Engineer to drive the evolution of our cloud-native, SaaS-based marketing and CRM platform serving banks and credit unions across the U.S. The role will focus on technical vision, scalability, re-architecture, and...
-
Principal Engineer
5 days ago
bangalore, India Coffeee.io Full timeRole- principal EngineerLocation- Bangalore (Hybrid)Experience- 10-12 yearsEmployment- Full timeAbout the RoleWe are looking for a highly skilled Principal Engineer with strong technical expertise andproven experience in engineering leadership. This role is ideal for someone who enjoys beinghands-on with coding while also managing and mentoring a small team....
