Lead Data Engineer

2 days ago

Hyderabad, Telangana, India Mobilution It Systems Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Location-Remote

Notice Period-Immediate-10 days

The Lead Data Engineer will architect, build, and operate scalable batch and real-time data pipelines on Google Cloud Platform. The role owns end-to-end data integration using Python, Apache Spark, Kafka, and GCP services like Dataflow, BigQuery, Cloud Composer, Cloud Functions, and Cloud Run. The engineer will ensure data quality, security, redaction/DLP, and robust CI/CD practices while optimizing performance and cost. They will collaborate with data scientists and analysts, design APIs with FastAPI/APIGEE, and guide deployments on GKE/Cloud Run. The position requires strong SQL skills, cloud migration experience, and excellent documentation and communication.

Required Qualifications

  • Expert-level Python for backend/data engineering
  • Hands-on Google Cloud Platform (Dataflow, BigQuery, Cloud Composer, Cloud Functions, Cloud Run, GCS, IAM)
  • Design and development of ETL/ELT pipelines (batch and streaming)
  • Apache Spark for large-scale processing
  • Apache Kafka for messaging/streaming
  • Orchestration with Airflow/Cloud Composer (DAG design, scheduling, monitoring)
  • Strong SQL with enterprise RDBMS (SQL Server, Oracle, PostgreSQL)
  • Git/GitHub and CI/CD for data projects
  • Deployments on GKE and Cloud Run; understanding of autoscaling and load balancers
  • API development with Python FastAPI; APIGEE proxy management
  • Data quality frameworks, validation rules, monitoring
  • Security: IAM roles/policies, data redaction, DLP
  • Data ingestion from varied sources; transformation and cleansing
  • Documentation of pipelines, data flows, and operations
  • Experience migrating on-prem data to cloud

Responsibilities

  • Design, develop, test, and maintain scalable ETL/ELT data pipelines in Python
  • Architect data solutions using Kafka, GCP services (Dataflow, BigQuery, Composer, Functions, Cloud Run/GKE, GCS, IAM), DBT, and related tools
  • Implement streaming and batch processing; ensure autoscaling, reliability, and cost efficiency
  • Develop and manage Airflow/Cloud Composer DAGs for orchestration
  • Build APIs with FastAPI and manage APIs via APIGEE; integrate load balancers when required
  • Ingest data from diverse sources; apply transformation, cleansing, and enrichment
  • Implement data quality checks, validation rules, observability, and monitoring
  • Apply security best practices including IAM, data redaction, and DLP
  • Write and optimize complex SQL for extraction, validation, and analytics
  • Manage code in GitHub; implement CI/CD pipelines for data workloads
  • Deploy and operate workloads on GKE and Cloud Run
  • Collaborate with data science/analytics teams to deliver fit-for-purpose data products
  • Support documentation of pipeline designs, data flows, and operational runbooks
  • Lead or contribute to on-prem to cloud data migration initiatives

  • Azure Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Azure Data Engineer

    4 days ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Senior Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India The Modern Data Company Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job InformationDate Opened11/07/2025IndustrySoftware ProductJob TypeFull timeCityHyderabadState/ProvinceTelanganaCountryIndiaZip/Postal Code500081About UsThe Modern Data Company is redefining data management for the AI era, transforming data from a technical challenge into a company's most powerful business asset. Modern's DataOS platform is the world's...

  • PeopleSoft To Workday Data Conversion Technical Lead

    5 hours ago

    Hyderabad, Telangana, India NTT DATA, Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job summaryThe PeopleSoft to Workday Data Conversion Technical Lead will be responsible for the technical design, development, and execution of the data migration from PeopleSoft to Workday. This hands-on role requires deep technical expertise in both PeopleSoft and Workday data structures and tools.The lead will manage the technical conversion team,...

  • Data Engineer Lead

    2 days ago

    Hyderabad, Telangana, India 6de69dcd-810e-48f0-b2ca-7f2997d6762e Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Company DescriptionCYTEL STATISTICAL SOFTWARE & SERVICES PRIVATE LIMITED is a professional organization based in Pune, Maharashtra, India. Operating out of the "Lohia-Jain IT Park" in Kothrud, the company is dedicated to providing innovative solutions in the statistical software and services domain. Known for its expertise and commitment to excellence, the...

  • Lead Data Engineer

    10 hours ago

    Hyderabad, Telangana, India Proclink Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    DescriptionProclink is looking for Data Engineer Lead.Experience :9+ years.Location :Hyderabad.Position OverviewWe are seeking an experienced Data Engineer with over 10 years of experience in designing, developing, and optimizing enterprise-scale data solutions.The ideal candidate will lead our data engineering strategy, architect cloud-based data platforms,...

  • Data Engineer Lead

    4 days ago

    Hyderabad, Telangana, India Proclink Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Proclinkis looking forData Engineer LeadExperience -9+ yearsLocation- HyderabadPosition OverviewWe are seeking an experiencedData Engineerwith over 10 years of experience in designing, developing, and optimizing enterprise-scale data solutions. The ideal candidate will lead our data engineering strategy, architect cloud-based data platforms, and mentor...

  • Lead Data Engineer

    3 hours ago

    Hyderabad, Telangana, India Argano Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    JOB DATA JOB TITLE: Lead - Data Engineer Argano is the world's largest global digital consultancy, exclusively connecting design and delivery for the transformation of high-performance business operations, extending our clients' commercial agility, profitability, customer experience, and growth. Our strategic consulting, bolstered by proprietary IP,...

  • Data Conversion Technical Lead

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Global Delivery Services Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    PeopleSoft to Workday Data Conversion Technical Lead Job summary The PeopleSoft to Workday Data Conversion Technical Lead will be responsible for the technical design, development, and execution of the data migration from PeopleSoft to Workday. This hands-on role requires deep technical expertise in both PeopleSoft and Workday data structures and tools....

  • Lead - Data Engineering

    1 week ago

    Hyderabad, Telangana, India Lera Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Position:Lead - Data EngineeringLocations: HyderabadExperience Level:7-10 yearsAbout UsLera Technologies is a future-focused, AI-led digital transformation company that empowers businesses to innovate and grow in today's fast-paced technology landscape. Our core strength lies in our flagship products like the 9X Data Platform which is a state-of-the-art...