Data Engineering Expert

2 weeks ago

Navi Mumbai, Maharashtra, India beBeeTechnical Full time ₹ 15,00,000 - ₹ 20,00,000

Job Title: Senior Technical Lead

Role Overview:

The Senior Technical Lead will be responsible for leading a team of technical professionals in designing, developing, and supporting multiple data engineering projects with heterogeneous data sources.

Key Responsibilities:

  • Architect and design multiple data engineering projects with heterogeneous data sources, produce/consume data to/from messaging queues like Kafka, push/pull data to/from REST APIs.
  • Support and enhance the in-house built Data Integration Framework, Data Replication Framework, Data Profiling & Reconciliation Framework.
  • Support, enhance, and lead data engineering initiatives, infrastructure, and architecture.
  • Engage/establish best standard practices with third-party data providers.
  • Translate complex technical subjects into terms that can be understood by both technical and non-technical audiences.
  • Provide direction and guidance to a team of 2 or more - solving problems and resolving conflicts.

Requirements:

  • 7+ years of experience using Data Integration Tools - Pentaho, Talend Or any other ETL/ELT tools.
  • 7+ years of experience using traditional databases like Postgres, MSSQL, Oracle.
  • 4+ years of experience using MPP databases like Vertica, Google BigQuery, Amazon Redshift.
  • 3+ years of experience in creating Entity-Relationship & Dimensional Data Model.
  • 2+ years of experience in Scheduler/Orchestration Tools Like Control-M, Autosys, Airflow, JAMS.
  • 2+ years of experience in leading a team of 2 or more.
  • Strong implementation experience with ETL/ELT Strategies - determining heavy lifting of data using Push Down Or Push Up methods.
  • Strong infrastructure administration knowledge on ELT/ETL Tools - Memory Management, Resource Management, User/Role Management, Performance improvement.
  • Strong implementation or working experience in any Code Versioning Tools.
  • Strong communication and documentation skills.
  • Experience in collaborating with multiple business units like Finance, Marketing, Sales.
  • Experience of working in Agile Delivery Model.

Desirable:

  • Experience in Data Visualization Tools like Tableau, Pentaho BA Tools.
  • Experience on the Hadoop ecosystem - Programmed or worked with key data components such as HIVE, Spark and Sqoop moving and processing terabyte level of data.
  • Digital Marketing/Web analytics or Business Intelligence a plus.
  • Understanding of Ad stack and data (Ad Servers, DSM, Programmatic, DMP, etc).
  • Knowledge of scripting languages such as Perl or Python.
  • Experience in the Linux environment is preferred but not mandatory.

Work Timings: 2 PM to 11 PM IST

Location: Navi Mumbai, India

  • Data Engineering Expert

    2 weeks ago

    Mumbai, Maharashtra, India beBeeDataEngineering Full time ₹ 8,00,000 - ₹ 15,00,000

    Job Title: Data Engineering ExpertWe are seeking a highly skilled Data Engineering Expert to join our team.

  • Lead Mechanical Engineer

    2 weeks ago

    Navi Mumbai, Maharashtra, India BDx Data Centers Full time

    Big Data Exchange (BDX), A leading Pan Asian hybrid, hyperscale and edge solutions provider with assets located in Hong Kong, Singapore, Guangzhou, Nanjing, Indonesia and expanding footprints in SEA regions. Our hybrid ecosystem provides colocation, Enterprise Services, private cloud services, disaster recovery services, and interconnect solutions across...

  • Data Center Systems Specialist

    2 weeks ago

    Navi Mumbai, Maharashtra, India beBeeData Center Expert Full time ₹ 15,00,000 - ₹ 28,00,000

    Job Overview","This role focuses on designing, configuring, and installing HVAC Controls and Electrical SCADA systems in data centers.","Key responsibilities include developing project scopes, managing budgets, and ensuring designs meet operational requirements.","Candidates should have a degree in engineering and 10+ years of experience with...

  • Expert Data Engineer

    7 days ago

    Mumbai, Maharashtra, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 35,00,000

    Job Title: Data Engineering ExpertYou will play a key role in designing, developing, and implementing data pipelines using Apache Spark and Databricks notebooks to process and transform large datasets.A strong understanding of data processing concepts, ETL/ELT pipelines, and big data architectures is required. You must have experience with SQL, relational...

  • Lead Data Engineer

    4 weeks ago

    Navi Mumbai, Maharashtra, India CrossTab IT Consulting Full time

    Experience: 6-8 years In depth understanding of spark framework and experience working with Spark SQL and Pyspark. Expert in Python programming language and common python libraries Strong analytical experience with database in writing complex queries, query optimization, debugging, user defined functions, views, indexes etc. Good Problem-Solving skills...

  • Senior Data Engineering Expert

    1 week ago

    Mumbai, Maharashtra, India beBeeDataEngineering Full time US$ 90,000 - US$ 1,20,000

    Job Title: Senior Data Engineering ExpertThe ideal candidate will be an expert in data engineering with a strong understanding of cloud-based platforms, particularly Databricks. They will have hands-on experience with Delta Lake, Genie, and ML Flow, as well as proficiency in Python, R, and other programming languages used in data processing.The selected...

  • Lead Data Engineer

    4 weeks ago

    Navi Mumbai, Maharashtra, India JobTravia Pvt. Ltd. Full time

    Job Title Lead Data Engineer Education Bachelor's degree in engineering (CS, IT, EXTC or equivalent) Experience: 6-8 years In depth understanding of spark framework and experience working with Spark SQL and Pyspark. Expert in Python programming language and common python libraries Strong analytical experience with database in writing complex queries,...

  • Lead Data Engineer

    2 weeks ago

    Navi Mumbai, Maharashtra, India JobTravia Pvt. Ltd. Full time US$ 90,000 - US$ 1,20,000 per year

    Job Title Lead Data EngineerEducation Bachelor's degree in engineering (CS, IT, EXTC or equivalent)Experience: 6-8 yearsIn depth understanding of spark framework and experience workingwith Spark SQL and Pyspark.Expert in Python programming language and common python librariesStrong analytical experience with database in writing complex queries,query...

  • Data Engineering Expert

    1 week ago

    Mumbai, Maharashtra, India beBeeEngineering Full time ₹ 50,000 - ₹ 13,00,000

    Job DescriptionWe are seeking a highly skilled Data Engineering Expert to lead our data team. This role will be responsible for designing, developing, and optimizing ETL/ELT data pipelines using Apache Spark (PySpark or Scala), AWS Glue, and Azure Data Factory.Key ResponsibilitiesData Pipeline Development: Design, develop, and optimize ETL/ELT data pipelines...

  • Senior Data Engineering Expert

    1 week ago

    Mumbai, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Data Engineer - ExpertCoforge Ltd. is seeking a Senior Data Engineer with expertise in designing and implementing robust data pipelines to steer business strategies.If you excel in tackling intricate data hurdles and enjoy guiding others, this opportunity could be your next career move. Job Overview:Develop and refine intricate ETL pipelines for processing...