Lead Data Engineer

16 hours ago

India People Prime Worldwide Full time

Important Note (Please Read Before Applying) 🚫 Do NOT apply if: You have less than 10 years or more than 12 years of total IT experience You do not have hands-on experience with Python and GCP Data Engineering projects You lack real-world experience in building and deploying ETL/ELT pipelines using GCP services (Dataflow, BigQuery, Composer, etc.) You have no exposure to Apache Spark, Kafka, or Airflow You are on a notice period longer than 30 days You are looking for a purely remote role (onsite/client engagement required) You come from a non-data background (e.g., testing, support, or non-engineering roles) Apply ONLY if you meet ALL mandatory skills and criteria above. Job Title: Lead Data Engineer - GCP + Python Location: Hyderabad Years of Experience: 10+Years About the Company Our client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future. Job Description : Key Responsibilities: ● Design, develop, test, and maintain scalable ETL data pipelines using Python. ● Architect the enterprise solutions with various technologies like Kafka, multi-cloud services, auto-scaling using GKE, Load balancers, APIGEE proxy API management, DBT, using LLMs as needed in the solution, redaction of sensitive information, DLP (Data Loss Prevention) etc. ● Work extensively on Google Cloud Platform (GCP) services such as: ○ Dataflow for real-time and batch data processing ○ Cloud Functions for lightweight serverless compute ○ BigQuery for data warehousing and analytics ○ Cloud Composer for orchestration of data workflows (on Apache Airflow) ○ Google Cloud Storage (GCS) for managing data at scale ○ IAM for access control and security ○ Cloud Run for containerized applications Required Skills: ● 10 to 12 years of hands-on experience in Python for backend or data engineering projects. ● Strong understanding and working experience with GCP cloud services (especially Dataflow, BigQuery, Cloud Functions, Cloud Composer, etc.). ● Solid understanding of data pipeline architecture, data integration, and transformation techniques. ● Experience in working with version control systems like GitHub and knowledge of CI/CD practices. ● Experience in Apache Spark, Kafka, Redis, Fast APIs, Airflow, GCP Composer DAGs. ● Strong experience in SQL with at least one enterprise database (SQL Server, Oracle, PostgreSQL, etc.). ● Experience in data migrations from on-premise data sources to Cloud platforms. Mandatory Skills Strong hands-on expertise in Python for backend and data engineering projects Proven experience in Google Cloud Platform (GCP) — especially Dataflow, BigQuery, Cloud Composer, Cloud Functions, GCS, and IAM Strong understanding of ETL/ELT pipeline design, architecture, and development Hands-on experience with Apache Spark, Kafka, and Airflow (GCP Composer DAGs) Experience in developing APIs using Python FastAPI framework Experience with GKE (Google Kubernetes Engine) and Cloud Run for deployment Strong SQL skills with enterprise databases (SQL Server / Oracle / PostgreSQL) Familiarity with NoSQL databases like MongoDB, Redis, or Bigtable Proficient in Git/GitHub and CI/CD pipeline deployments for data projects Experience in data migration from on-premise sources to Cloud platforms Solid knowledge of data quality, validation, monitoring, and data governance best practices

  • ▷ High Salary! Data

    3 weeks ago

    Remote, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data & AI Engineer Lead to join our team in Remote, Karntaka (IN-KA), India (IN). Job Duties: Role Overview The Data...

  • Data Engineer

    1 week ago

    India NTT DATA Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Req ID: 343254NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Remote, Karnātaka (IN-KA), India (IN). "Key Responsibilities: Design and implement...

  • Data Analytics Lead

    6 days ago

    Delhi, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Analytics Lead to join our team in Delhi, Delhi (IN-DL), India (IN). Experience: 8+ Years Location: Delhi NCR...

  • Data Engineer

    15 hours ago

    Bengaluru, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Bangalore, Karntaka (IN-KA), India (IN). Job Duties: Snowflake - working experience...

  • Eucloid - Senior/Lead Data Engineer - Python

    4 weeks ago

    Chennai, India Eucloid Data Solutions Full time

    Job Description Job Description Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and...

  • Data Engineer

    6 days ago

    Mumbai, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Mumbai, Mahrshtra (IN-MH), India (IN). Job Duties: Key Responsibilities: . Design...

  • Crest Data Systems

    4 weeks ago

    Ahmedabad, India Crest Data Full time

    Job Description Company Overview Crest Data is a leading provider of data center solutions and engineering/marketing services in the areas of Networking/SDN, Storage, Security, Virtualization, Cloud Computing, and Big Data / Data Analytics. The team has extensive experience in building and deploying various Data Center products from Cisco, VMware, NetApp,...

  • Data Engineer

    4 weeks ago

    Bengaluru, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Bangalore, Karntaka (IN-KA), India (IN). Design and implement tailored data...

  • Lead Data Engineer

    3 weeks ago

    India ATOM Systems Private Limited Full time

    Job DescriptionName of the position :  Data Engineer Location :   Coimbatore / Remote  No. of resources needed:  01  Mode:  Contract to Hire Years of experience:  15+ Years  About the Role We are seeking a highly skilled and driven Data Engineering Lead to lead our data engineering team. The ideal candidate combines strong...

  • Lead Data Engineer

    3 weeks ago

    India ATOM Systems Private Limited Full time

    Job Description Name of the position : Data Engineer Location : Coimbatore / Remote No. of resources needed: 01 Mode: Contract to Hire Years of experience: 15+ Years About the Role We are seeking a highly skilled and driven Data Engineering Lead to lead our data engineering team. The ideal candidate combines strong leadership and technical expertise with the...