Lead Data Engineer

3 weeks ago

india QuEST Global Full time

Applicants can submit CV direct to Email : Sunil Chandran ;


Experience : 6 - 12 Years


Responsibilities:


  • Design, develop, and maintain scalable and efficient data pipelines to extract, transform, and load (ETL) data from various sources into data lakes and data warehouses.
  • Design and develop microservices to
  • Collaborate with data scientists, analysts, and cross functional teams to design data models, database schemas and data storage solutions.
  • Implement data integration and data quality processes to ensure the accuracy and reliability of data for analytics and reporting.
  • Optimize data storage, processing, and querying performance for large-scale datasets.
  • Enable advanced analytics and machine learning capabilities on the data platform.
  • Continuously monitor and improve data quality and data governance practices.
  • Stay up to date with the latest data engineering trends, technologies, and best practices.


Requirements:


  • Bachelor’s degree in computer science, Engineering, or a related field.
  • 8+ of proven experience in data engineering, data warehousing, and ETL processes.
  • Proficiency in data engineering tools and technologies such as SQL, Python, Spark, Hadoop, DBT, Airflow, Apache Kafka and Presto.
  • Solid experience with table formats such as Delta Lake or Apache Iceberg
  • Design and development experience with batch and real time streaming infrastructure and workloads.
  • Solid experience with implementing data lineage, data quality and data observability for Big data workflows.
  • Solid experience designing and developing microservices and distributed architecture.
  • Hands-on experience with GCP ecosystem and data lakehouse architectures.
  • Strong experience with implementing Databricks.
  • Strong experience with container technologies such as Docker, Kubernetes.
  • Strong understanding of data modeling, data architecture, and data governance principles.
  • Excellent experience with DataOps principles and test automation.
  • Familiarity with data processing and querying using distributed systems and NoSQL databases.
  • Ability to optimize and tune data processing and storage for performance and cost-efficiency.
  • Excellent problem-solving skills and the ability to work on complex data engineering challenges.
  • Strong communication and collaboration skills to work effectively with cross-functional teams.
  • Previous experience mentoring and guiding junior data engineers is a plus.
  • Relevant certifications in data engineering or cloud technologies are desirable.


Nice to have:

  • Experience working in a Data Mesh architecture.
  • Supply Chain domain experience.


Special requirements


  • Support US Data platform teams
  • Monitor and manage platform infrastructure
  • Assist with platform deployment and maintenance
  • Support Insights, Control Tower, and other production needs
  • For Data pipeline automation and efficient data integration


Skills required: Strong experience with platform support and operations, cloud infrastructure management. Strong understanding of Insights, Control Tower and other products that data platform supports. Expertise with data engineering.

  • Operations Specialist

    1 month ago

    India Data Engineer Academy Full time

    Job Title: Operations Analyst Location : RemoteCompany Overview:Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary to...

  • Operations Specialist

    4 weeks ago

    india Data Engineer Academy Full time

    Job Title: Operations Analyst Location : Remote Company Overview: Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary...

  • Operations Specialist

    4 weeks ago

    india Data Engineer Academy Full time

    Job Title: Operations Analyst Location : Remote Company Overview: Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary...

  • Operations Specialist

    6 days ago

    India Data Engineer Academy Full time

    Job Title: Operations Analyst Location : Remote Company Overview: Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary...

  • Data Engineering Lead

    6 days ago

    India Chryselys Full time

    Job Description:We are currently seeking a skilled and experienced individual for the position of Data Engineering Lead. As the Data Engineering Lead, you will have a significant impact on the successful implementation of data projects for our clients. Your proficiency in data engineering and data operations will be crucial in ensuring project completion...

  • Lead Data Engineer

    6 days ago

    India Decision Foundry Full time

    Lead Data Engineer (Salesforce Cloud):Experience: 5-10 years of experience in data engineering and the Salesforce ecosystem.We are looking for a skilled Senior Data Engineer to join our team and have a key role in creating, constructing, and advising on solutions within the Salesforce ecosystem. The perfect candidate should have a solid background in data...

  • Lead Data Engineer

    6 days ago

    India Orion Innovation Full time

    As a Lead Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure. You will lead a team of data engineers to ensure the successful delivery of projects and contribute to the overall data strategy. The ideal candidate will have extensive experience in cloud-based data technologies, particularly in Azure,...

  • Lead Data Engineer

    3 weeks ago

    india Orion Innovation Full time

    As a Lead Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure. You will lead a team of data engineers to ensure the successful delivery of projects and contribute to the overall data strategy. The ideal candidate will have extensive experience in cloud-based data technologies, particularly in Azure,...

  • Lead Data Engineer

    4 weeks ago

    india Orion Innovation Full time

    As a Lead Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure. You will lead a team of data engineers to ensure the successful delivery of projects and contribute to the overall data strategy. The ideal candidate will have extensive experience in cloud-based data technologies, particularly in Azure,...

  • Data Engineer

    2 months ago

    india dentsu Full time

    The purpose of this role is to maintain, improve, clean and manipulate data in the business’s operational and analytics databases. The Data Engineer works with the business’s software engineers, data analytics teams, data scientists and data warehouse engineers in order to understand and aid in the implementation of database requirements, analyse...

  • Lead data engineer

    4 weeks ago

    india USEReady Full time

    Useready is hiring for Lead Data Engineer(Bangalore/Mohali/Gurugram) Job Description Mentor a team of developers in building and maintaining high-quality, scalable, and secure data pipelines and analytics applications on the Databricks platform Work closely with data engineers, business stakeholders to understand their needs and design and develop...

  • Lead Data Engineer

    4 weeks ago

    india Digitas India Full time

    Role- Lead Data Engineer Experience - 6 +Years Locations- Mumbai/Pune/Ahmedabad Overview: As a Lead Data Engineer, you will play a pivotal role in designing, implementing, and managing complex data pipelines within Google Cloud Platform (GCP) & AWS . Your expertise in integrating SAP, Salesforce and other cloud databases and marketing platforms , will be...

  • Lead Data Engineer

    3 weeks ago

    india Helius Technologies Full time

    Work Location: Hyderabad Work Mode : 5 days Work from Office Work Timings : 9 am to 6 pm Client: Singapore based Job Type: 1 year renewable contract (Long Term) Kindly revert with below details in addition to updated resume. Full Name as per the Passport or Aadhar Card : Current Salary Exp Salary Notice period (availability) Reason for Change Ok for 1...

  • Snowflake Data Engineer

    4 weeks ago

    india Lapis Data Analytics Full time

    Company Description Lapis Data Analytics is a dynamic and innovative data engineering company founded in 2020 and located in Chennai. Our primary focus is on designing, building, and maintaining robust data pipelines that convert raw data into actionable insights. With a team of seasoned experts and cutting-edge technology, we deliver customized data...

  • Lead Data Engineer

    3 weeks ago

    india Decision Foundry Full time

    Lead Data Engineer (Salesforce Cloud): Experience: 5-10 years of experience in data engineering and the Salesforce ecosystem. We seek an experienced Senior Data Engineer to join our team and play a pivotal role in designing, building, and consulting solutions within the Salesforce ecosystem. The ideal candidate should have a strong background in data...

  • Lead Data Engineer

    3 weeks ago

    india Decision Foundry Full time

    Lead Data Engineer (Salesforce Cloud): Experience: 5-10 years of experience in data engineering and the Salesforce ecosystem. We seek an experienced Senior Data Engineer to join our team and play a pivotal role in designing, building, and consulting solutions within the Salesforce ecosystem. The ideal candidate should have a strong background in data...

  • Azure Lead Data Engineer

    1 month ago

    india IntraEdge Full time

    As Azure Lead Data Engineer, you will have to take over the key activities of designing, coding, and implementing data solutions in the platform. With a focus on leveraging DBT for data transformation and modelling, as well as expertise in MDM tools, you will play a pivotal role in architecting scalable and performant data pipelines and warehouses. You will...

  • Data Engineering Lead

    1 month ago

    india Chryselys Full time

    Job Description: As a Data Engineering Lead, you will play a critical role in driving the delivery for our clients data projects. Leveraging your expertise in data engineering and data operations, you will work closely with clients to ensure that the project delivery on time with quality and meeting the requirements. This role requires a deep understanding...

  • Team Lead- Data Engineering

    5 days ago

    India Concentrix Full time

    Experience: 6 Years - 10 Years Job Location: Bengaluru (Hybrid- currently remote) Notice Period: Immediate/ 30 Days (Looking for someone to start from May or Mid June) Shift Timings: 02:00 PM to 11:00 PM (IST)/ 03:00 PM to 12:00 AM (IST) Job Description: We are seeking an experienced and talented Data Engineering Team Lead who can manage a high-performing...

  • Team Lead- Data Engineering

    4 weeks ago

    india Concentrix Full time

    Experience: 6 Years - 10 Years Job Location: Bengaluru (Hybrid- currently remote) Notice Period: Immediate/ 30 Days (Looking for someone to start from May or Mid June) Shift Timings: 02:00 PM to 11:00 PM (IST)/ 03:00 PM to 12:00 AM (IST) Job Description: We are seeking an experienced and talented Data Engineering Team Lead who can manage a...