Cynosure Corporate Solutions | Lead Data Engineer

4 days ago


india Cynosure Corporate Solutions Full time

Roles and Responsibilities:


  • Working with clients to understand their data.
  • Based on the understanding you will be building the data structures and pipelines.
  • You will be working on the application from end to end collaborating with UI and other development teams.
  • You will be working with various cloud providers such as Azure & AWS.
  • You will be engineering data using the Hadoop/Spark ecosystem.
  • You will be responsible for designing, building, optimizing and supporting new and existing data pipelines.
  • Orchestrating jobs using various tools such Oozie, Airflow, etc.
  • Developing programs for cleaning and processing data.
  • You will be responsible for building the data pipelines to migrate and load the data into the HDFS either on-prem or in the cloud.
  • Developing Data ingestion/process/integration pipelines effectively.
  • Creating Hive data structures, metadata and loading the data into data lakes / Bigdata warehouse environments.
  • Optimized (Performance tuning) many data pipelines effectively to minimize cost.
  • Code versioning control and git repository is up to date.
  • You should be able to explain the data pipeline to internal and external stakeholders.
  • You will be responsible for building and maintaining CI/CD of the data pipelines.


Preferred Qualifications:


  • Bachelor’s degree in computer science or related field.
  • Minimum of 5+ years working experience with Spark, Hadoop eco systems.
  • Minimum of 4+ years working experience on designing data streaming pipelines
  • Minimum experience of 3+ years on NoSQL and Spark Streaming.
  • Proven experience with big data ecosystem tools such as Sqoop, Spark, SQL, API, Hive, Oozie, Airflow, etc..
  • Solid experience in all phases of SDLC with 10+ years of experience (plan, design, develop, test, release, maintain and support)
  • Hands-on experience using Azure’s data engineering stack.
  • Should have implemented projects using programming languages such as Scala or Python.
  • Working experience on SQL complex data merging techniques such as windowing functions etc..
  • Hands-on experience with on-prem distribution tools such as Cloudera/Horton Works/MapR.



  • india Cynosure Corporate Solutions Full time

    Roles and Responsibilities:Working with clients to understand their data.Based on the understanding you will be building the data structures and pipelines.You will be working on the application from end to end collaborating with UI and other development teams.You will be working with various cloud providers such as Azure & AWS.You will be engineering data...


  • India Cynosure Corporate Solutions Full time

    Cynosure Corporate Solutions is seeking a highly skilled Data Engineer to lead our big data solutions team. This role requires expertise in designing, building, and optimizing complex data pipelines using Hadoop/Spark ecosystem.About the Role:We are looking for an experienced professional who can work collaboratively with cross-functional teams to deliver...


  • india Wavicle Data Solutions Full time

    Job Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key Responsibilities:...


  • india Wavicle Data Solutions Full time

    Job Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...


  • india Wavicle Data Solutions Full time

    Job Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key Responsibilities:...

  • Lead data engineer

    3 weeks ago


    India Wavicle Data Solutions Full time

    Job Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, Py Spark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...

  • Lead Data Engineer

    4 weeks ago


    India Wavicle Data Solutions Full time

    Job Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...

  • Lead Data Engineer

    4 weeks ago


    India Wavicle Data Solutions Full time

    Job Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...


  • India Wavicle Data Solutions Full time

    Wavicle Data Solutions is seeking an experienced Lead Data Engineer to play a pivotal role in architecting and implementing data solutions.About the RoleThis position offers a competitive salary of $180,000 - $220,000 per annum, commensurate with experience. Key ResponsibilitiesDesign and develop scalable and efficient data pipelines using Python and...


  • india Data Warehouse Engineer Full time

    Experience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.


  • India Data Warehouse Engineer Full time

    Experience : 2- 5 Years Primary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills. Good to have skills: Power BI, Databricks, Python.


  • India Data Warehouse Engineer Full time

    About the RoleWe are seeking an experienced Data Warehouse Engineer to join our team. In this role, you will design, develop, and maintain data warehouses that support business intelligence and analytics initiatives.


  • india Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...


  • india Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement: Design and architect integration solutions to connect various enterprise applications, systems, and databases. Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications. Utilize Azure Integration Services such as Azure Logic Apps,...


  • India Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement: Design and architect integration solutions to connect various enterprise applications, systems, and databases. Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications. Utilize Azure Integration Services such as Azure Logic...


  • India ITI Data Full time

    ITI Data is seeking a highly skilled Cloud Data Engineering Lead to spearhead the design and development of scalable data pipelines on AWS. With a strong background in PySpark, Python, and AWS services, you will be responsible for building robust data integration solutions that enable data-driven insights across various internal and external sources.This is...


  • india IRIS-Corporate Solutions Full time

    Company Profile IRIS Corporate Solutions Private Limited is a new-age HR solutions company focused on preparing Millennials, Gen Z, and Gen Alpha for the future. With three decades of expertise in HR services, we deliver quality interventions to create a skilled workforce through recognized HR delivery models. Our technology-driven approach offers...


  • india IGT Solutions Full time

    Role: Lead Data Engineer Primary Skills: Python, Pyspark, AWS Glue, EMR, Lambda, Snowflake, ETL, PLSQL Experience: - 7+ Years Location: - Remote Notice period: - Immediate Joiner Job Overview : We are seeking a skilled and experienced Lead Data Engineer to join our data engineering team. The ideal candidate will have over 8 years of hands-on experience...

  • Lead Data Engineer

    6 days ago


    India IGT Solutions Full time

    Role: Lead Data Engineer Primary Skills: Python, Pyspark, AWS Glue, EMR, Lambda, Snowflake, ETL, PLSQL Experience: - 7+ Years Location: - Remote Notice period: - Immediate Joiner Job Overview : We are seeking a skilled and experienced Lead Data Engineer to join our data engineering team. The ideal candidate will have over 8 years of hands-on...


  • India ITI Data Full time

    We are seeking an experienced PySpark Data Engineer to join our team at ITI Data. As a key member of our team, you will design and build scalable data solutions for one of our Fortune 500 Client programs.About the Role:The ideal candidate will have extensive experience in working with Hadoop, NoSQL databases, and Spark framework on Python. You will be...