Sr Data Engnieer Lead

Attention All Data Folks We are Hiring Sr Data Engineering Lead Azure AWS in Indore MP Hybrid Model Below is the JD for the reference Sr Data Engineering Lead Indore MP Hybrid 3 Days a Week Full Time Exp Level- 8-15 years About the Role Ccube is seeking a highly skilled Data Engineering Lead to architect build and scale our enterprise-grade data infrastructure This role demands a strong technical foundation in distributed data systems deep hands-on experience with modern big data and cloud technologies and proven leadership in building and mentoring high-performing data engineering teams The ideal candidate will shape the data engineering roadmap drive innovation and ensure robust scalable and efficient data solutions across the organization Key Responsibilities Data Strategy Architecture Define the end-to-end data engineering strategy and technical vision aligned with Ccube s business goals Architect scalable high-performance and cost-efficient data platforms on AWS Azure or GCP Design and optimize data lakehouse medallion architectures integrating batch and streaming pipelines Spark Kafka Delta Lake Iceberg Hudi etc Build reusable frameworks for data ingestion transformation and orchestration across heterogeneous systems Data Engineering Execution Lead the development and optimization of ETL ELT pipelines using PySpark Scala SQL and Airflow or equivalent orchestration tools Oversee the implementation of real-time streaming solutions leveraging Kafka Kinesis or Pub Sub Guide the team in integrating structured semi-structured and unstructured data sources Drive adoption of DataOps and DevOps best practices CI CD for data pipelines automated testing and monitoring Cloud Infrastructure Design and manage cloud-native data solutions AWS Glue EMR Redshift Snowflake BigQuery Databricks etc Optimize infrastructure for scalability performance and cost using Terraform CloudFormation or Pulumi Lead initiatives for data platform modernization and cloud migration strategies Governance Security Observability Define and enforce standards for data quality lineage governance and metadata management e g Great Expectations Apache Atlas or Collibra Implement robust data security compliance and privacy frameworks aligned with industry standards GDPR HIPAA etc Establish observability frameworks for data pipelines logging monitoring and anomaly detection Leadership Collaboration Lead and mentor a team of senior and mid-level data engineers fostering a culture of excellence ownership and innovation Collaborate cross-functionally with AI ML Analytics and Product Engineering teams to enable data-driven decision-making Evaluate emerging technologies e g VectorDBs GraphDBs RAG frameworks and drive their adoption for advanced data-driven use cases Represent the data engineering practice in architecture reviews and executive technology forums Required Qualifications Experience 10 years in data engineering and architecture roles with 3 years in technical leadership or data platform lead roles Technical Expertise Deep proficiency in Spark PySpark Scala SQL and distributed data processing Proven hands-on work in cloud data platforms - AWS Glue EMR Redshift GCP Dataflow BigQuery or Azure Synapse Data Factory Experience with workflow orchestration Airflow Dagster Prefect and containerization Docker Kubernetes Expertise in modern data storage systems - Snowflake Databricks Iceberg Hudi or Delta Lake Familiarity with Graph Databases Neo4j AWS Neptune and Vector Databases Pinecone Weaviate Milvus for AI RAG systems Exposure to data observability data mesh and feature store frameworks Leadership Strong people management mentorship and cross-functional collaboration skills Demonstrated success in building or scaling a data engineering function or CoE Center of Excellence Certifications Preferred AWS Certified Data Analytics - Specialty GCP Professional Data Engineer Databricks Certified Data Engineer Snowflake Architect Why Join Ccube Build the Future - Be part of a mission-driven company shaping the next generation of AI and digital solutions Collaborative Culture - Work with kind brilliant people who value transparency experimentation and integrity Career Mobility - We invest in your learning and promote from within Compensation Equity - Competitive base commission stock options Professional Growth - Annual stipend for certifications courses and industry events Are you ready to help companies unlock the power of technology and build something extraordinary in the process Feel Free to reach out to me at rahul for more details Powered by JazzHR