Lead Software Engineer-Data Engineering

Career Area:

Job Description:

Your Work Shapes the World at Caterpillar Inc.

When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other.  We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here – we make it happen, with our customers, where we work and live. Together, we are building a better world, so we can all enjoy living in it.

About the Role

We are seeking a highly skilled Software Engineering Lead to lead a team of engineers in building Caterpillar's next-generation Digital Manufacturing Data Platform. This platform is central to our Future of Manufacturing initiative—designed to unify and operationalize data across design, engineering, production, and supply chain operations.

The ideal candidate will possess deep expertise in Big Data, Data Warehousing, real-time data movement, and Snowflake-based architectures. You will architect and deliver scalable, secure, and intelligent data platforms that enable advanced analytics, AI, and digital twin capabilities across global manufacturing ecosystems.

Key Responsibilities

Team Leadership & Management

• Lead, mentor, and manage a team of data engineers and platform developers.
• Foster a culture of technical excellence, collaboration, and continuous learning.
• Drive Agile practices and ensure timely delivery of high-quality solutions.

Technical Strategy & Architecture

• Architect and oversee the development of scalable, secure, and resilient data platforms.
• Design and implement near real-time data movement and streaming architectures using tools like Kafka, Spark, and cloud-native services.
• Establish best practices in data modelling, ETL/ELT, data governance, and metadata management.

Data Engineering & Snowflake Expertise

• Lead the development of robust data pipelines for ingestion, transformation, and delivery using Snowflake, dbt, and cloud-native tools.
• Optimize data storage, retrieval, and processing for performance, reliability, and cost-efficiency.
• Implement data quality frameworks, lineage tracking, and schema evolution strategies.

Big Data & Data Warehousing

• Build and maintain large-scale data lakes and data warehouses for structured and unstructured data.
• Design scalable data architectures to support manufacturing analytics, predictive maintenance, and supply chain optimization.

Cloud & Platform Engineering

• Leverage Azure and AWS services for data ingestion, transformation, and analytics.
• Deploy software using CI/CD tools (Azure DevOps preferred, Jenkins, AWS CloudFormation).
• Ensure platform scalability, security, and operational readiness across global deployments.

AI & Advanced Analytics Enablement

• Collaborate with Data Science and AI teams to operationalize ML models and analytics workflows.
• Promote integration of AI capabilities into data engineering pipelines (e.g., GenAI, MCP, ATA).
• Support real-time analytics and edge AI use cases in manufacturing environments.

Stakeholder Engagement

• Partner with product managers, manufacturing SMEs, and business leaders to understand requirements and deliver impactful data solutions.
• Communicate technical concepts to non-technical audiences and influence strategic decisions.

Must-Have Skills

• Proven experience in Big Data processing and Data Warehousing.
• Expertise in building end-to-end near real-time data pipelines for OLTP & OLAP.
• Strong architecture exposure for building robust, scalable Data Platforms.
• Deep expertise in Snowflake, SQL, NoSQL, and distributed data systems.
• Experience with data transformation tools (dbt, Apache Spark, Azure Data Factory).
• Strong analytical skills and solid knowledge of computer science fundamentals.
• Deep exposure to Azure and AWS cloud platforms.
• Good understanding of AI concepts and latest developments (Gen AI, MCP, ATA, etc.).

Nice-to-Have Skills

• Knowledge of the NVIDIA ecosystem and its applications in data and AI.
• Experience building production-ready AI solutions and integrating with MLOps workflows.
• Familiarity with modern data visualization and BI tools (e.g., Power BI, Tableau, Looker).

Qualifications

• Bachelor's or master's degree in computer science, Engineering, or related field.
• 10+ years of experience in data engineering, with at least 2+ as team lead
• Demonstrated success in managing engineering teams and delivering complex data solutions.
• Excellent communication, leadership, and stakeholder management skills.
• This position requires working onsite five days a week.

Posting Dates:

Caterpillar is an Equal Opportunity Employer.  Qualified applicants of any age are encouraged to apply

Not ready to apply? Join our Talent Community.