GCP Lead Data Engineer
19 hours ago
Lead Data Engineer – Python & GCPLocation - 10+ YearsJob Overview:We are seeking an experienced Lead Data Engineer with strong expertise in Python and Google Cloud Platform (GCP). The ideal candidate will lead the end-to-end data engineering lifecycle—from requirement gathering and solution design to development, deployment, and post-delivery support. This role involves designing scalable ETL/ELT pipelines, architecting cloud-native solutions, implementing data ingestion & transformation processes, and ensuring data quality across systems.Experience Level:10+ years of relevant IT experience in data engineering and backend development.Key Responsibilities:Design, develop, test, and maintain scalable ETL/ELT data pipelines using Python.Architect enterprise-grade data solutions using technologies such as Kafka, GKE, multi-cloud services, load balancers, APIGEE, DBT, LLMs, and DLP tools.Work extensively with GCP services, including:Dataflow – real-time & batch processingCloud Functions – serverless computeBigQuery – data warehousing & analyticsCloud Composer – workflow orchestration (Airflow)GCS – scalable storageIAM – access control & securityCloud Run – containerized workloadsBuild APIs using Python FastAPI.Work with Big Data and processing technologies:Apache Spark, Kafka, Airflow, MongoDB, Redis/BigtablePerform data ingestion, transformation, cleansing, and validation to ensure high data quality.Implement and enforce data quality checks, monitoring, and validation rules.Collaborate with data scientists, analysts, and engineering teams to understand data needs and deliver solutions.Use GitHub for version control and support CI/CD deployments.Write complex SQL queries for relational databases such as SQL Server, Oracle, and PostgreSQL.Document data pipeline designs, architecture diagrams, and operational procedures.Required Skills:10+ years of hands-on experience with Python in data engineering or backend development.Strong working knowledge of GCP services (Dataflow, BigQuery, Cloud Functions, Cloud Composer, GCS, Cloud Run).Deep understanding of data pipeline architecture, ETL/ELT processes, and data integration patterns.Experience with Apache Spark, Kafka, Airflow, FastAPI, Redis/Bigtable.Strong SQL skills with at least one enterprise RDBMS (SQL Server, Oracle, PostgreSQL).Experience in on-prem to cloud data migrations.Knowledge of GitHub and CI/CD best practices.Good to Have:Experience with Snowflake.Hands-on knowledge of Databricks.Familiarity with Azure Data Factory (ADF) or other Azure data tools.
-
Data Engineer
3 weeks ago
Erode, India Whatjobs IN C2 Full timeJob Title: Data Engineer Location: Remote Position: Full-time Exp- Must Have 8+ Years Position Overview We are currently seeking a Data Engineer to join our technology team. In this role, you will work on building data-driven products that sit atop cloud-based data infrastructure. The ideal candidate is passionate about technology, software design, data...
-
Engineering Manager
7 days ago
Erode, India Tamara Full time🚀 Data Engineering Manager II: Real-Time Data & Experimentation Platform (Remote, India)About TamaraTamara is Saudi Arabia's first fintech unicorn 🦄 and the leading financial super-app in the GCC, with a mission to empower dreams through customer-centric financial solutions. We partner with giants like SHEIN, IKEA, Amazon, and Jarir, serving millions...
-
Enterprise Data Architect
2 days ago
Erode, India Ironbook AI Full timeJob Title: AWS Data Architect (5+ Years Experience) Job Summary: The AWS Data Architect will design, implement, and optimize scalable data architectures on AWS, ensuring robust data pipelines, data storage, and governance to meet business needs. The candidate will work closely with cross-functional teams to translate business requirements into technical...
-
Backend Lead
1 day ago
Erode, India Schbang Full timeBackend Lead: 4 -5 YearsLocation: Lower Parel, MumbaiWe are seeking an experienced Backend Lead to join our dynamic team. The ideal candidate will have a solid foundation of at least 4-5 years in Node.JS, PHP, Wordpress, MongoDB. This role requires proficiency in cloud platforms such as AWS, Azure, or GCP, and the ability to design, develop, and maintain...
-
Senior data engineer
1 day ago
Erode, India Nityo Infotech Full timeSenior Data Engineer Position Overview We are seeking an experienced Senior Data Engineer with 8-12 years of hands-on experience to join our team on-site. This role requires a self-starter who can architect and implement robust data foundation layers while providing technical leadership and guidance to the engineering team. The ideal candidate will have deep...
-
Lead Databricks Analyst
19 hours ago
Erode, India Rialtes Full timeLocation: RemoteEmployment Type: Full-TimeOverviewRialtes is seeking a Databricks Support Analyst to manage platform operations, ensure reliability of clusters and jobs, and support development of data workflows. Role requires hands-on experience with Databricks administration, SQL-based development, and basic Python.Key ResponsibilitiesPrimary: Platform...
-
Associate Director
5 days ago
Erode, India Brownloop Full timeAbout Brownloop:Brownloop is a US-based fintech company focused on leveraging data, analytics, and AI solutions to help Private Equity firms make smarter investment decisions. We work with some of the largest and most reputed Private Equity companies in the world. We plan to expand our presence into India by setting up our core data competency and product...
-
Associate director
3 days ago
Erode, India Brownloop Full timeAbout Brownloop: Brownloop is a US-based fintech company focused on leveraging data, analytics, and AI solutions to help Private Equity firms make smarter investment decisions. We work with some of the largest and most reputed Private Equity companies in the world. We plan to expand our presence into India by setting up our core data competency and product...
-
Data Engineer
7 days ago
Erode, India Canopus Infosystems - A CMMI Level 3 Company Full timeJob Title: Data Engineer (Analyst) Experience: 2+ Years Location: Remote About the Role: We are looking for an experienced Data Engineer (Analyst) who can design, build, and optimize data models and pipelines to support business intelligence and performance analytics. The ideal candidate should have a strong understanding of SQL, cloud data...
-
Ai Systems
3 weeks ago
Erode, India Whatjobs IN C2 Full timeAbout the Role Flight Crew International (FCI) is seeking a technically skilled and solutions-oriented AI Systems Integration & Automation Engineer to lead the development of our next generation of intelligent workflows and digital systems. This role will combine OpenAI integration, aviation software connectivity (e.G., Avianis by Portside), and enterprise...
