Lead Data Engineer
3 hours ago
Position: Lead Data Engineer
Location: Bengaluru
Experience: 10+ years
Job Summary:
We are seeking an experienced Senior Data Engineer with a strong background in PySpark, Spark, and Big Data technologies. The ideal candidate will have over five years of hands-on experience designing, building, and optimizing large-scale data processing systems and pipelines.
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines and ETL processes using PySpark and Spark.
- Perform data wrangling, transformation, and integration across diverse data sources.
- Optimize the performance and reliability of data processing workflows.
- Implement data quality frameworks to ensure data accuracy and integrity.
- Develop and maintain data architectures and data models to support business needs.
- Collaborate with cross-functional teams including data analysts, software engineers, and business stakeholders to understand data requirements.
- Stay updated with the latest advancements in Big Data tools, frameworks, and best practices.
- Mentor and guide junior data engineers in technical and professional growth.
Required Skills and Qualifications:
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
- 5+ years of experience in data engineering roles.
- Expertise in PySpark and Spark for large-scale data processing.
- Strong understanding of Big Data ecosystems and technologies (e.g., Hadoop, Hive).
- Hands-on experience with ETL development, data modeling, and data warehousing.
- Proficiency in programming languages such as Python, Scala, or Java.
- Experience with cloud platforms like AWS, Azure, or Google Cloud.
- Strong problem-solving skills and a detail-oriented mindset.
- Excellent communication and collaboration skills.
Preferred Qualifications:
- Experience with containerization and orchestration tools like Docker and Kubernetes.
- Knowledge of SQL and relational database management systems.
- Familiarity with data orchestration tools like Airflow, Prefect, or similar.
-
Data & AI Engineer Lead
5 days ago
Bengaluru, Karnataka, India NTT DATA Full time ₹ 12,00,000 - ₹ 36,00,000 per yearFramework Design & Architecture Architect a metadata-driven, Python/Spark-based framework for automated data validation across high-volume production datasets. Define DQ rule templates for completeness, integrity, conformity, accuracy, and timeliness. Establish data quality thresholds, escalation protocols, and exception workflows. Automation & Integration...
-
Lead Data Engineer
3 hours ago
Bengaluru, Karnataka, India, Karnataka Dr. Martens plc Full timeSO, WHAT'S THE STORY? As a Lead Data Engineer, you will play a key role in the development and maintenance of the organization's data infrastructure. Working within a multi-disciplined team led by the Senior Manager - Data Engineer, you will focus on building and optimizing scalable data pipelines and supporting the delivery of high-quality, reliable data...
-
Lead GCP Data Engineer
3 hours ago
Bengaluru, Karnataka, India, Karnataka Impetus Full timeJob Title: Lead Data Engineer – GCP (BigQuery • Composer • Python • PySpark)Location: BengaluruExperience: 8+ years (data engineering / analytics engineering), with previous lead responsibilitiesAbout the Role:You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform. You will manage a team of data...
-
AWS Lead Data engineer
3 hours ago
Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full timeJob Summary: In this key leadership role, you will lead the development of foundational components for a Lakehouse architecture on AWS and drive the migration of existing data processing workflows to the new Lakehouse solution. You will work across the Data Engineering organisation to design and implement scalable data infrastructure and processes using...
-
Enterprise Data Architect
3 hours ago
Bengaluru, Karnataka, India, Karnataka Aptus Data Labs Full timeEnterprise Data Architect – Job DescriptionLocation: Bangalore (Hybrid)Experience: 15+ years in Data Engineering & Data PlatformsEmployment Type: Full-time About Aptus Data LabsAptus Data Labs is a global Data Engineering and AI solutions partner helping enterprises build modern, scalable, and intelligence-driven organizations. With deep expertise across...
-
Data Engineer
2 weeks ago
Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per yearDesign and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack. Provide thought leadership by recommending the most appropriate technologies and solutions for a given use case, covering the entire spectrum from the application layer...
-
Associate Lead Data Scientist
2 weeks ago
Bengaluru, Karnataka, India Astar Data Full time ₹ 8,00,000 - ₹ 24,00,000 per yearSigmoid Analytics is a leading Data solutions company backed by Sequoia Capital. We offer best in- end-to-end data value chain spanning across Data Science, Data Engineering and Data Ops. With data and technology at the core of our solutions, we are solving some of the toughest problems out there. Our culture is modelled around expertise and mutual respect...
-
IT Global Team lead
3 hours ago
Bengaluru, Karnataka, India, Karnataka Continental Full time-Lead and manage the global IT team for data engineering that develops all technical artefacts as code, implemented in professional IDEs, with full version control and CI/CD automation. This team combines both lakehouse modeling of common and business use ase artefacts & semantics, as well as generalist data integration & metadata services.-Ensure...
-
Sr. Data Engineer Data Science
2 weeks ago
Bengaluru, Karnataka, India NTT DATA Full time ₹ 12,00,000 - ₹ 24,00,000 per yearReq ID: 342869NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Sr. Data Engineer Data Science to join our team in Bangalore, Karnātaka (IN-KA), India (IN). ...
-
Technical Lead-Data Engineer
3 hours ago
Bengaluru, Karnataka, India, Karnataka Best Infosystems Ltd. Full timeTechnical Lead-Data Engineer_Full-Time_Bangalore/Pune/Navi Mumbai/Noida/Hyderabad/ChennaiJob Title: Technical Lead-Data EngineerLocation: Bangalore/Pune/Navi Mumbai/Noida/Hyderabad/ChennaiJob Type: Full-TimeExperience: 8+ yearsQualifications: Bachelor’s/relevant exp - with Snowpro CertificationJob Description:Must Have Skills:•Overall – 8+...