SquareShift - Lead Data Engineer - Python/SQL

3 days ago

Chennai, India SQUARESHIFT TECHNOLOGIES PRIVATE LIMITED Full time

Overview : We are seeking a highly skilled Lead Data Engineer to architect, build, and scale enterprise-grade data platforms and pipelines.The ideal candidate has deep expertise in modern data engineering, cloud-native services, real-time data ingestion & CDC frameworks.This role involves leading complex data initiatives, guiding engineering teams, and collaborating with cross-functional stakeholders to deliver robust, scalable, and secure data solutions.Key Responsibilities : Data Architecture & Pipeline Development : - Design and build scalable ETL/ELT pipelines using Python / PySpark.- Architect and implement batch and real-time data pipelines using Kafka, Kafka Connect, and Debezium.- Develop and optimize data ingestion workflows across multiple databases (MySQL, Postgres, MongoDB).- Build and manage Apache Airflow DAGs for end-to-end pipeline orchestration.Cloud Data Engineering : - Lead data platform development on GCP (BigQuery, Data Proc, Data Stream ).- Implement data ingestion, orchestration, and transformation at scale using Pub/Sub, Cloud Functions/Cloud Run, Dataflow, and GKE.- Own data warehouse design and optimization across Snowflake and BigQuery.Requirements : Required Skills & Experience : Core Technical Skills : - Strong proficiency in Python, SQL, PySpark.- Hands-on expertise with Kafka, Kafka Connect, Debezium, Airflow, Databricks.- Deep experience with BigQuery, Snowflake, MySQL, Postgres, MongoDB.- Solid understanding of vector data stores and search indexing.- Knowledge of GCP services like Big Query, Cloud Functions, Cloud Run, Data Flow, Data Proc, Data Stream, etc.Good to have : Certifications : - GCP Professional Data Engineer.- Elastic Certified Engineer.AI : - Gemini Enterprise.- Vertex AI Agent Builder.- ADK.Non-Technical & Leadership Skills : - Communication : Exceptional verbal and written communication skills, with the ability to articulate complex technical concepts to both technical and non-technical audiences.- Mentorship & Coaching : Proven experience in mentoring junior and mid-level engineers, fostering a culture of continuous learning and growth.- Problem-Solving : Strong analytical and debugging skills, with a proactive approach to identifying and resolving technical roadblocks.- Ownership & Accountability : Demonstrates a high level of responsibility for project outcomes, system reliability, and code quality.- Agile Proficiency : Deep understanding and practical experience with Agile methodologies (Scrum/Kanban).- Stakeholder Management : Ability to effectively manage expectations and build consensus across different teams.Qualifications : - Bachelors or Masters degree in Computer Science, Engineering, or a related field (or equivalent practical experience).- Typically 7+ years of progressive experience in data engineering, with 2+ years in a technical leadership or lead engineer role. (ref:hirist.tech)

  • Lead Data Engineer

    2 days ago

    Chennai, Tamil Nadu, India SquareShift Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Lead Data Engineer - Job DescriptionOverviewWe are seeking a highly skilled Lead Data Engineer to architect, build, and scale enterprise-grade data platforms and pipelines. The ideal candidate has deep expertise in modern data engineering, cloud-native services, real-time data ingestion & CDC frameworks. This role involves leading complex data initiatives,...

  • Lead Data Engineer

    4 days ago

    Chennai, India SquareShift Technologies Full time

    Job Description Lead Data Engineer - Job Description Overview We are seeking a highly skilled Lead Data Engineer to architect, build, and scale enterprise-grade data platforms and pipelines. The ideal candidate has deep expertise in modern data engineering, cloud-native services, real-time data ingestion & CDC frameworks. This role involves leading complex...

  • Lead Data Engineer

    4 days ago

    Chennai, India SquareShift Technologies Full time

    Lead Data Engineer - Job Description Overview We are seeking a highly skilled Lead Data Engineer to architect, build, and scale enterprise-grade data platforms and pipelines. The ideal candidate has deep expertise in modern data engineering, cloud-native services, real-time data ingestion & CDC frameworks. This role involves leading complex data initiatives,...

  • Lead Data Engineer

    2 days ago

    Chennai, Tamil Nadu, India SquareShift Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job InformationDate Opened11/24/2025Job TypeFull timeIndustryTechnologyWork Experience5+CityChennaiState/ProvinceTamil NaduCountryIndiaZip/Postal Code600001Job DescriptionLead Data Engineer - Job DescriptionOverviewWe are seeking a highly skilled Lead Data Engineer to architect, build, and scale enterprise-grade data platforms and pipelines. The ideal...

  • SquareShift - Technical Lead - DevOps & Cloud Security

    2 weeks ago

    Chennai, India SQUARESHIFT TECHNOLOGIES PRIVATE LIMITED Full time

    Job Title : Tech Lead DevOps & Cloud InfrastructureLocation : ChennaiExperience Required : 8 to 12 YearsEmployment Type : Full-Time / PermanentAbout the Role : We are looking for an experienced and strategic DevOps Tech Lead to drive our cloud infrastructure and CI/CD initiatives.This role is ideal for a seasoned professional with 8 - 12 years of experience...

  • Python with SQL

    4 days ago

    Chennai, India Vega Intellisoft Full time

    EXP 6+ years Python,Sql,Nosql n - Strong Python development experience, in an enterprise environment - Python library knowledge including packages such as FastAPI and Dash - Experience with both SQL and NoSQL database platforms - An understanding of data structures and common data transformation methodologies - A drive to deliver with an ability to own a...

  • Senior Data Engineer

    2 weeks ago

    Chennai, India YO IT CONSULTING Full time

    Job Description : Title : Senior Data EngineerExperience : 6 to 8 yearsLocation : ChennaiMust Have : - SQL- Databricks- PySpark- PythonRole And Responsibilities : The Data Engineer will be responsible for designing and developing data engineering solutions, creating and maintaining data models, building Extract Transform Load (ETL) pipelines, implementing...

  • Eucloid - Senior Data Engineer - Python/SQL/ETL

    2 weeks ago

    Chennai, India Eucloid Data Solutions Full time

    Job Description Job Description : is looking for a Senior Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and...

  • Python & SSIS Data Engineer

    7 days ago

    Chennai, Tamil Nadu, India AABM Cloud Data Solutions Full time ₹ 10,00,000 - ₹ 42,00,000 per year

    MANDATORY PREVIOUS WORK EXPERIENCE IN SSIS AND PYTHON. (Candidates without this experience in the past will NOT be considered for this role)JOB RESPONSIBILITIES5+ years previous work experience as a Data Engineer.Design and build data pipelines using SSIS, ADF, Talend, Informatica, Data Stage, or Ab Initio to move and transform data reliably.Manage and...

  • Eucloid - Senior Data Engineer - Python/SQL/ETL

    3 weeks ago

    Chennai, India Eucloid Data Solutions Pvt Ltd Full time

    Job Description : is looking for a Senior Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and building of...