Data Engineering Lead

2 weeks ago

Kollam, Kerala, India beBeeData Full time ₹ 1,20,00,000 - ₹ 2,50,00,000
Job Description:

We are seeking a highly skilled Senior Data Engineer to join our team. The ideal candidate will have experience in designing, developing, and implementing large-scale data processing systems using AWS technologies.

  • Study the existing technology landscape and understand the current data integration framework and do impact assessments for new project requirements.
  • Develop Spark jobs using Scala for new project requirements.
  • Enhance existing Spark jobs for ongoing product enhancements.
  • Perform performance tuning of Spark jobs, stress testing, etc.
  • Create new data pipelines for developed/enhanced Spark jobs using AWS Lambda or Apache Airflow.
  • Responsible for database design process, logical design, physical design, star schema, snowflake schema, etc.
  • Analyze data processing, integration, modeling, and reporting requirements, and define data loading strategies considering volume, data types, frequency, and analytics specifications.
  • Evaluate optimal balance between cost and performance.
Required Skills:
  • 7-9 years of IT experience.
  • 5+ years of AWS-related project experience.
  • Good to have Associate Level and Professional Level AWS Certification.
  • In-depth knowledge of following AWS Services: S3, EC2, EMR, Serverless, Athena, AWS Glue, Lambda, Step Functions.
  • Cloud Databases: AWS Aurora, Singlestore, RedShift, Snowflake.
  • Big Data: Hadoop, Hive, Spark, YARN.
  • Programming Language: Scala, Python, Shell Scripts, PySpark.
  • Operating System: Any flavor of Linux, Windows.
  • Strong SQL skills.
  • Orchestration Tools: Apache Airflow.
Benefits:
  • Competitive salary.
  • Bonus scheme.
  • Opportunities for professional growth and development.
  • A collaborative and dynamic work environment.

  • Data Scientist/ML Engineer

    3 weeks ago

    Kollam, Kerala, India Quant-data Full time

    We're Hiring: Machine Learning Engineer / Data Engineer (Remote | Full-Time) Build AI-powered credit decisioning systems on Microsoft AzureWe're looking for a Machine Learning Engineer / Data Engineer with 5+ years of experience to join our AI-driven credit lending platform team. In this role, you'll design and deploy scalable ML solutions that power loan...

  • Cloud-Based Data Engineer Lead

    2 weeks ago

    Kollam, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Lead Data Engineer PositionJob Summary:This senior-level position is available for a highly skilled and experienced data engineer who can lead the development and implementation of scalable data pipelines using cloud-based technologies.Responsibilities:Design, develop, and optimize ETL/ELT data pipelines using Apache Spark (PySpark or Scala), AWS Glue, and...

  • Senior Data Engineering Lead

    1 week ago

    Kollam, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    We are seeking an experienced Technical Project Manager with a background in data engineering.The ideal candidate will have hands-on experience in managing end-to-end projects involving data pipelines, ETL processes, and data integration workflows.A key aspect of this role is overseeing the delivery of data platforms, including AI, BI, and analytics...

  • Data Engineering Technical Lead

    1 week ago

    Kollam, Kerala, India beBeeProjectManager Full time ₹ 17,50,000 - ₹ 22,50,000

    About this role: We are seeking a skilled Technical Project Manager to lead and deliver projects in data engineering and analytics. You will manage cross-functional teams to execute data platform, pipeline, and analytics initiatives, ensuring alignment with business goals and timely delivery.Key Responsibilities:Plan, execute, and deliver technical projects...

  • Lead Data Architect

    5 days ago

    Kollam, Kerala, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    We are looking for an experienced data engineer to create, enhance and manage our data processing infrastructure.The ideal candidate will have a deep understanding of designing and implementing efficient data pipelines using cloud-based technologies like Azure Data Factory, Databricks and Synapse Analytics.Key Responsibilities:Create scalable data...

  • Data engineer

    2 weeks ago

    Kollam, Kerala, India Centrilogic Full time

    Data EngineerPurpose:Over 15 years, we have become a premier global provider of multi-cloud management, cloud-native application development solutions, and strategic end-to-end digital transformation services.Headquartered in Canada and with regional headquarters in the U. S. and the United Kingdom, Centrilogic delivers smart, streamlined solutions to...

  • Lead Data Infrastructure Engineer

    5 days ago

    Kollam, Kerala, India beBeeDatabase Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title:Senior Database AdministratorOur ideal candidate will possess a strong background in database administration and performance tuning. They should have expertise in SQL, particularly with PostgreSQL, MSSQL, and Snowflake environments. Additionally, they should have experience working with Amazon RDS and familiarity with data engineering tools like...

  • Data Engineering Leader

    2 weeks ago

    Kollam, Kerala, India beBeeDataEngineeringLeader Full time ₹ 40,00,000 - ₹ 50,00,000

    Job Title: Data Engineering LeaderAs a Data Engineering Leader, you will play a key role in driving the strategic direction of our data engineering capabilities. You will be responsible for leading cross-functional teams to design and implement large-scale data processing systems, leveraging cloud-native technologies such as AWS.The ideal candidate will have...

  • Lead Data Platform Engineer

    2 weeks ago

    Kollam, Kerala, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job Title: Lead Data Platform Engineer - Product Development Specialist.A renowned corporation is seeking a skilled data engineer to manage the development and deployment of advanced data pipelines. The ideal candidate will possess expertise in cloud infrastructure, database management, and API design.Responsibilities include leading technical teams,...

  • Data Engineering Professional

    6 days ago

    Kollam, Kerala, India beBeeDataEngineering Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job Title: Data Engineering ProfessionalWe are seeking an experienced Data Engineer to lead our team in leveraging cutting-edge technologies and innovative solutions for data processing and analytics.Key Responsibilities:Design, develop, and maintain large-scale data systems using Postgres SQLCreate efficient data models to support business intelligence and...