Lead Data Engineer – AWS

2 weeks ago

Cochin, Kerala, India Experion Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Job Location: KochiRemoteTrivandrum

Job Purpose

  • We are seeking an experienced Senior Data Engineer to lead the development of a scalable data ingestion framework while ensuring high data quality and validation. The successful candidate will also be responsible for designing and implementing robust APIs for seamless data integration. This role is ideal for someone with deep expertise in building and managing big data pipelines using modern AWS-based technologies, and who is passionate about driving quality and efficiency in data processing systems.

Job Description / Duties and Responsibilities

  • Architect, develop, and maintain end-to-end data ingestion framework for extracting, transforming, and loading data from diverse sources
  • Use AWS services (Glue, Lambda, EMR, ECS, EC2, Step Functions) to build scalable, resilient automated data pipelines
  • Develop and implement automated data quality checks, validation routines, and error-handling mechanisms
  • Establish comprehensive monitoring, logging, and alerting systems for data quality issues
  • Architect and develop secure, high-performance APIs for data services integration
  • Create thorough API documentation and establish standards for security, versioning, and performance
  • Work with business stakeholders, data scientists, and operations teams to understand requirements
  • Participate in sprint planning, code reviews, and agile ceremonies
  • Contribute to CI/CD pipeline development using GitLab

Job Specification / Skills and Competencies

  • 5+ years experience in data engineering with analytical platform development focus
  • Proficiency in Python and/or PySpark
  • Strong SQL skills for ETL processes and large-scale data manipulation
  • Extensive AWS experience (Glue, Lambda, Step Functions, S3)
  • Familiarity with big data systems (AWS EMR, Apache Spark, Apache Iceberg)
  • Database experience with DynamoDB, Aurora, Postgres, or Redshift
  • Proven experience designing and implementing RESTful APIs
  • Hands-on CI/CD pipeline experience (preferably GitLab)
  • Agile development methodology experience
  • Strong problem-solving abilities and attention to detail
  • Excellent communication and interpersonal skills
  • Ability to work independently and collaboratively
  • Capacity to quickly learn and adapt to new technologies
  • 10+ years total experience

Any Additional Information/Specifics

  • Bachelor's/Master's in Computer Science, Data Engineering or related field (preferred)
  • Experience with additional AWS services (Kinesis, Firehose, SQS)
  • Familiarity with data lakehouse architectures and modern data quality frameworks
  • Experience with proactive data quality management in multi-cluster environments
  • Adhere to Information Security Management policies and procedures

  • Lead Data Engineer

    1 week ago

    Cochin, Kerala, India Careersteer services private limited Full time ₹ 10,00,000 - ₹ 23,00,000 per year

    We are seeking a highly skilled Lead Data Engineer with strong hands-on experience in AWS, Python, and PySpark. The ideal candidate will design, develop, and manage scalable data pipelines and architectures, lead a team of data engineers, and ensure data availability, reliability, and performance to support analytics, reporting, and business intelligence...

  • aws ai engineer

    2 weeks ago

    Cochin, Kerala, India Aabasoft Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    We are seeking talented AWS AI Engineers / Developers to join our team in building innovative agentic AI solutions. This role focuses on implementing intelligent, automated workflows that enhance clinical data processing and decision-making, leveraging the power of AWS services and cutting-edge open-source AI frameworks. Experience in healthcare or Life...

  • Machine Learning Engineer – AWS

    1 day ago

    Cochin, Kerala, India DBiz ai solutions Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Summary:We are seeking a highly skilled and motivated Machine Learning Engineer with a strong foundation in programming and machine learning, hands-on experience with AWS Machine Learning services (especially SageMaker), and a solid understanding of Data Engineering and MLOps practices. You will be responsible for designing, developing, deploying, and...

  • Senior Data Engineer

    3 days ago

    Cochin, Kerala, India Experion Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Location: KochiRemoteTrivandrumExperience5+ YearsJob PurposeWe are seeking an experienced Senior Big Data Engineer to take a leading role in designing, building, and managing the data driven apps for one of our customers. You will be responsible for understanding evolving customer needs and translating them into robust, scalable, and cost-effective data...

  • Lead Data Architect

    5 days ago

    Cochin, Kerala, India Tamcherry Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Description: Data and AI Lead ArchitectWe are seeking an experienced Data and AI Lead Architect with 12–15 years of expertise in large-scale data and cloud architecture, particularly on AWS or Google Cloud, who has successfully supported enterprise customers across the US and UK/Europe. The ideal candidate will be based in Kochi (Kerala in general) or...

  • Data Engineer

    2 weeks ago

    Cochin, Kerala, India VGreenTek Full time ₹ 70,000 - ₹ 1,20,000 per year

    Job Role: Data EngineerExperience: 5+ yearsMode: Work from office / Hybrid (Kochi)Job Type: Full-time, PermanentNotice Period: daysJob Description:This role is a hands-on position responsible for the build and continued evolution of the data platform, business applications and integration tools. We are looking for a hands-on engineer that can recommend best...

  • Data Engineer

    3 days ago

    Cochin, Kerala, India NOV Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    This role is part of the Digital Technology Services team at NOV. As a Data Engineer, you'll support data and analytics initiatives by helping to build and maintain data pipelines, data infrastructure, and operationalize analytics applications. You'll have the opportunity to learn a wide variety of technologies and work closely with experienced engineers,...

  • AWS DevOps Engineer

    5 days ago

    Cochin, Kerala, India Tedora Hacker Hcademy Full time ₹ 96,000 - ₹ 3,60,000 per year

    AWS Trainer jobTedora Hacker Academy Position: AWS (Amazon Web Services) Trainer Organization: Tedora Hacker Academy Location: 4th Floor, Indo Persian NJK Square, Valanjambalam, Sahodaran Ayyappan Rd, Ernakulam South, Ernakulam, Kerala – Working Days: Monday to SaturdayAbout UsTedora Hacker Academy is a leading professional training institute based in...

  • Senior Data engineer

    2 weeks ago

    Cochin, Kerala, India Milestone Technologies, Inc. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role Overview: Seeking an experienced Senior Data Engineer to lead data ingestion, transformation, and optimization initiatives using the modern Apache and Azure data stack. The role involves working on scalable pipelines, large-scale distributed systems, and data lake management. Core Responsibilities:Build and manage high-volume data pipelines using...

  • AWS Cloud Engineer

    5 days ago

    Cochin, Kerala, India Tedora Hacker Hcademy Full time ₹ 8,00,000 - ₹ 30,00,000 per year

    AWS Trainer jobTedora Hacker Academy Position: AWS (Amazon Web Services) TrainerOrganization: Tedora Hacker AcademyLocation: 4th Floor, Indo Persian NJK Square, Valanjambalam, Sahodaran Ayyappan Rd, Ernakulam South, Ernakulam, Kerala – 682016Working Days: Monday to SaturdayAbout UsTedora Hacker Academy is a leading professional training institute based in...