[Only 24h Left] Lead Data Engineer

Job Title Sr Data Engineering Lead Location Indore MP Hybrid 3 Days a Week Employment Type Full-Time About the Role Ccube is seeking a visionary Senior Data Engineering Lead to spearhead and lead our entire Data Management practice In this pivotal role you will architect build and scale our enterprise-grade data infrastructure enabling data-driven decision-making across the organization This position calls for a strong technical foundation in distributed data systems deep hands-on experience with modern big data and cloud technologies and a proven track record of leading and mentoring high-performing data engineering teams The ideal candidate will define the data engineering roadmap drive innovation and deliver robust scalable and efficient data solutions that power Ccube s growth and success Key Responsibilities Data Strategy Architecture Define the end-to-end data engineering strategy and technical vision aligned with Ccube s business goals Architect scalable high-performance and cost-efficient data platforms Preferably on AWS Azure or GCP Design and optimize data Lakehouse medallion architectures integrating batch and streaming pipelines Spark Kafka Delta Lake Iceberg Hudi etc Build reusable frameworks for data ingestion transformation and orchestration across heterogeneous systems Data Engineering Execution Lead the development and optimization of ETL ELT pipelines using PySpark Scala SQL and Airflow or equivalent orchestration tools Oversee the implementation of real-time streaming solutions leveraging Kafka Kinesis or Pub Sub Guide the team in integrating structured semi-structured and unstructured data sources Drive adoption of DataOps and DevOps best practices CI CD for data pipelines automated testing and monitoring Cloud Infrastructure Design and manage cloud-native data solutions AWS Glue EMR Redshift Snowflake BigQuery Databricks etc Lead initiatives for data platform modernization and cloud migration strategies Governance Security Observability Define and enforce standards for data quality lineage governance and metadata management e g Great Expectations Apache Atlas or Collibra Implement robust data security compliance and privacy frameworks aligned with industry standards GDPR HIPAA etc Establish observability frameworks for data pipelines logging monitoring and anomaly detection Leadership Collaboration Lead and mentor a team of senior and mid-level data engineers fostering a culture of excellence ownership and innovation Collaborate cross-functionally with AI ML Analytics and Product Engineering teams to enable data-driven decision-making Evaluate emerging technologies e g VectorDBs GraphDBs RAG frameworks and drive their adoption for advanced data-driven use cases Represent the data engineering practice in architecture reviews and executive technology forums Required Qualifications Experience 10 years in data engineering and architecture roles with 3 years in technical leadership or data platform lead roles Technical Expertise Deep proficiency in Spark PySpark Scala SQL and distributed data processing Strong experience with big data ecosystems Hadoop Hive Presto Trino Delta Lake etc Proven hands-on work in cloud data platforms - AWS Glue EMR Redshift GCP Dataflow BigQuery or Azure Synapse Data Factory Experience with workflow orchestration Airflow Dagster Prefect and containerization Docker Kubernetes Expertise in modern data storage systems - Snowflake Databricks Iceberg Hudi or Delta Lake Exposure to data observability data mesh and feature store frameworks Leadership Strong people management mentorship and cross-functional collaboration skills Demonstrated success in building or scaling a data engineering function or CoE Center of Excellence Certifications Preferred AWS Certified Data Analytics - Specialty GCP Professional Data Engineer Databricks Certified Data Engineer Snowflake Architec Powered by JazzHR