Data Engineering Lead

2 weeks ago

Chennai, Tamil Nadu, India beBeeDataEngineering Full time ₹ 22,00,000 - ₹ 23,00,000
Data Engineering Lead Position

We are seeking a highly skilled and experienced Data Engineering Lead to spearhead the development of a cutting-edge data ingestion framework.

Key Responsibilities:
  • Data Ingestion Framework Development:
    • Achieve scalability by designing, developing, and maintaining an end-to-end data ingestion framework that efficiently extracts, transforms, and loads data from diverse sources.
    • Leverage AWS services like AWS Glue, Lambda, EMR, ECS, EC2, and Step Functions to build robust, resilient, and automated data pipelines.
  • Data Quality & Validation:
    • Develop and implement comprehensive data quality checks, validation routines, and error-handling mechanisms to ensure the accuracy and integrity of incoming data.
    • Establish proactive monitoring, logging, and alerting systems to promptly identify and resolve data quality issues.
  • API Development:
    • Design and implement secure, high-performance APIs to facilitate seamless integration of data services with external applications and internal systems.
    • Create detailed API documentation and establish standards for API security, versioning, and performance optimization.
    Required Skills and Qualifications:
    • Experience & Technical Skills:
      • Professional Background: A minimum of 5 years of relevant experience in data engineering, emphasizing analytical platform development.
      • Programming Skills: Proficiency in Python and/or PySpark, SQL for ETL processes and large-scale data manipulation.
      • AWS Expertise: Extensive experience using AWS services including AWS Glue, Lambda, Step Functions, and S3 for data ingestion frameworks.
      • Data Platforms: Familiarity with big data systems (e.g., AWS EMR, Apache Spark, Apache Iceberg) and databases like DynamoDB, Aurora, Postgres, or Redshift.
      • API Development: Proven experience in designing and implementing RESTful APIs and integrating them with external and internal systems.
      • CI/CD & Agile: Hands-on experience with CI/CD pipelines (preferably with GitLab) and Agile development methodologies.
    • Soft Skills:
      • Strong problem-solving abilities and attention to detail.
      • Excellent communication and interpersonal skills with the ability to work independently and collaboratively.
      • Capacity to quickly learn and adapt to new technologies and evolving business requirements.

  • Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    HiRole - Azure Data engineerLocation - Chennai, Gurugram (5days Onsite)Full timeImmediate joiner Only (15days Notice period only)Technical Skills:o Proficiency in SQL for complex queries, database design, and optimization.o Strong experience with PySpark for data transformation and processing.o Hands-on experience with Databricks for building and managing...

  • Data Engineer

    5 days ago

    Chennai, Tamil Nadu, India NTT DATA North America Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer - Azure to join our team in Chennai, Tamil Nādu (IN-TN), India (IN)."Job Duties: Key Responsibilities:•  ...

  • Data Engineering Lead

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeDataEngineeringLead Full time ₹ 15,00,000 - ₹ 28,00,000

    Job Title: Data Engineering LeadJob Summary:As a highly experienced data engineering lead, you will be responsible for spearheading the development and implementation of modern data pipelines using Snowflake. This role requires strong technical leadership skills to mentor and guide junior team members.Required Skills and Qualifications:Expertise in...

  • Data Engineer Lead

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeEngineering Full time US$ 1,80,000 - US$ 2,50,000

    Lead Data EngineerWe are seeking an accomplished and seasoned Data Engineering professional to lead our data team in delivering impactful solutions within the healthcare and medical technology sectors.This role involves driving the development of scalable pipelines using Databricks, Azure Data Factory, and ETL tools for efficient data ingestion,...

  • data engineering lead

    3 weeks ago

    Chennai, Tamil Nadu, India Randstad Full time

    At least 5 years' experience as a Senior or Lead Data Engineer Proven and demonstratable experience of Python, AWS, and SQL/ Oracle Bachelor's honours degree preferably in computer science, or engineering field, or demonstrable technical experience in a related engineering discipline At least 3 years of ETL experience using flat files to transposed...

  • Data Engineering Lead

    1 week ago

    Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 13,80,147 - ₹ 24,45,294

    Key Responsibilities:We are seeking a highly skilled data engineer to lead the development and maintenance of robust data pipelines. The successful candidate will have a strong background in computer science and data engineering with experience working with Python, SQL, PySpark, and bash scripts.The ideal candidate will have 3+ years of experience in...

  • Senior Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Eucloid Data Solutions Full time US$ 1,50,000 - US$ 2,00,000 per year

    About The Role:Eucloid is looking for a senior Data Engineer with hands-on expertise in Databricks to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may include starting from up- stream and down-stream...

  • Lead Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Vijay Venkatesh ( Proprietor of Cynosure Corporate Solutions ) Full time

    Job DescriptionAbout the RoleWe are seeking a highly skilled and experienced Senior / Lead Data Engineer to design, develop, and maintain scalable, reliable, and efficient data pipelines and ETL solutions. The role requires strong expertise across multi-cloud environments, modern data warehousing platforms, programming languages, and data orchestration...

  • Lead Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Ixceed Solutions Full time

    About the Role :We are seeking a highly skilled and experienced Lead Data Engineer/Scala Developer with strong hands-on expertise in Scala programming, Apache Spark, Databricks, Python, and PySpark. The ideal candidate will lead the design, development, and deployment of scalable big data solutions, working closely with cross-functional teams to deliver...

  • Lead Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time

    About the Role : We are seeking a highly skilled and experienced Senior / Lead Data Engineer to design, develop, and maintain scalable, reliable, and efficient data pipelines and ETL solutions. The role requires strong expertise across multi-cloud environments, modern data warehousing platforms, programming languages, and data orchestration tools. You will...