Lead Data Engineer

3 weeks ago


india Recro Full time

Position: Data Engineer(Lead)


Location: WFO-Bangalore


Experience Level: 8+ years


Job Description:

Summary:

We're seeking a highly skilled and experienced Big Data Architect to join our innovative and dynamic team. You'll play a pivotal role in designing, building, and maintaining our scalable big data infrastructure. The ideal candidate will be a passionate problem-solver with a proven track record of developing robust data pipelines and architecting data products across diverse architectures.

Responsibilities:

  • Design and architect big data solutions leveraging distributed platforms (e.g., Spark, EMR) to meet complex business needs.
  • Partner with data scientists, analysts, and software engineers to understand data requirements and propose innovative data solutions.
  • Develop, implement, and manage high-performing data pipelines for ingesting, transforming, and delivering data into our data warehouse and data lake.
  • Champion data quality by implementing robust data cleansing, transformation, and enrichment processes.
  • Continuously monitor and optimize data pipelines for peak performance, scalability, and cost-effectiveness.
  • Integrate security best practices into data pipelines and ensure data protection measures are in place.
  • Troubleshoot and debug complex data pipeline issues, performance bottlenecks, and data quality problems.
  • Advocate for and contribute to improvements in the overall data engineering process.
  • Foster a collaborative environment by mentoring and developing other data engineers in adopting best practices.

Attributes & Qualifications:

Technical Skills:

  • 8+ years of experience in developing and architecting scalable big data applications or solutions.
  • Expertise in distributed processing frameworks like Spark (Python, Scala).
  • Experience with cloud platforms for data engineering (AWS S3, Managed Airflow, EMR/EC2, IAM etc. preferred).
  • Experience with data pipeline orchestration tools (Airflow, Jenkins) and CI/CD practices.
  • Proficiency in SQL databases (Presto, Snowflake) and data warehousing concepts.
  • Experience architecting data products in streaming, serverless, and microservices architectures.
  • Experience with Data Lakehouse architecture (e.g., Databricks Delta Lake, Medallion).
  • Strong understanding of data governance, data modeling, and data quality frameworks.
  • Familiarity with reporting and analytical tools (Tableau, Quicksight).

Soft Skills:

  • Excellent problem-solving skills with a keen eye for detail.
  • Strong communication and collaboration skills across technical and non-technical audiences.
  • Ability to work independently and as part of a fast-paced, Agile/Scrum development team.
  • Passion for data, a continuous learning mindset, and a willingness to learn new technologies.

  • Data Engineer Lead

    1 month ago


    india Booth and Partners Pte Ltd Full time

    Job Description This is a remote position. About the Client: They understand your pain. In fact, they've been there. It’s the very reason they built the industry’s first automated data analytics and reporting platform for pharmacy benefit organizations. After spending more than 30 years building manual reports for Payers, PBMs, TPAs, Consultants,...

  • Data Engineer

    1 month ago


    india dentsu Full time

    The purpose of this role is to maintain, improve, clean and manipulate data in the business’s operational and analytics databases. The Data Engineer works with the business’s software engineers, data analytics teams, data scientists and data warehouse engineers in order to understand and aid in the implementation of database requirements, analyse...


  • india IntraEdge Full time

    As Azure Lead Data Engineer, you will have to take over the key activities of designing, coding, and implementing data solutions in the platform. With a focus on leveraging DBT for data transformation and modelling, as well as expertise in MDM tools, you will play a pivotal role in architecting scalable and performant data pipelines and warehouses. You will...

  • Lead Data Engineer

    4 weeks ago


    india Solera, Inc. Full time

    About the job Who We Are Solera, DealerSocket is a global leader in data and software services that strives to transform every touchpoint of the vehicle lifecycle into a connected digital experience. In addition, we provide products and services to protect life’s other most important assets: our homes and digital identities. Today, Solera processes over...

  • Lead Data Engineer

    1 month ago


    India Pro5.ai (formerly Mangtas) Full time

    Job Description : Technical delivery lead on platform and project builds for building and maintaining optimised and highly available data pipelines that facilitate data management and data analytics solutions. The role will drive the development of data processing frameworks and data architectural approaches that scale to handle the business’s growing...

  • Lead Data Engineer

    1 month ago


    India Pro5.ai (formerly Mangtas) Full time

    Job Description: Technical delivery lead on platform and project builds for building and maintaining optimised and highly available data pipelines that facilitate data management and data analytics solutions.The role will drive the development of data processing frameworks and data architectural approaches that scale to handle the business’s growing...

  • Aptus Data Labs

    1 day ago


    india Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • Lead Data Engineer

    3 days ago


    india Akivna Technologies Pvt. Ltd. Full time

    Company Description Akivna Technologies Pvt. Ltd. is an IT company based in Gurugram. We specialize in Data Engineering, Data Science, Data Analytics, DevOps, and Product Development. Our team is passionate about technology and we strive to deliver innovative and high-quality solutions to our clients. Job Description: We are seeking a skilled and...

  • Lead Data Engineer

    3 days ago


    India Akivna Technologies Pvt. Ltd. Full time

    Company DescriptionAkivna Technologies Pvt. Ltd. is an IT company based in Gurugram. We specialize in Data Engineering, Data Science, Data Analytics, DevOps, and Product Development. Our team is passionate about technology and we strive to deliver innovative and high-quality solutions to our clients.Job Description:We are seeking a skilled and experienced...

  • Technical Lead

    2 weeks ago


    india Crest Data Systems Full time

    The Technical Lead for AI/ML plays a critical role in driving the technical direction and execution of artificial intelligence and machine learning projects within the organization. This position combines strong technical expertise in AI/ML with leadership skills to guide a team of engineers and data scientists in developing innovative AI/ML solutions. The...


  • india K2 Data & Analytics Full time

    Job Description : We are on the lookout for an enthusiastic ML Engineer with over five years of experience to be part of our team, which is focused on developing cutting-edge Video Ad Data technologies. The perfect fit for this role would be someone with a solid grasp of machine learning algorithms and a background in designing and executing large-scale...

  • Lead Data Engineer

    14 hours ago


    india EXL Full time

    Must have: Experience: 5+ years 5/6+ years of experience in Data Engineering, Architecture & Solutioning areas in large-scale distributed systems 3+ years of experience across Cloud Platforms, AWS preferred. 2+ years of experience in creating solutions for clients/products Hands on coding experience in Oracle / Teradata / SQL Server platform. Hands on...

  • Lead Data Engineer

    1 month ago


    india Tredence Full time

    MUST HAVE • Minimum of 4+ years of Azure DE experience • In-depth technical knowledge of tools like Azure Data Factory, Databricks, Azure Synapse, SQL DB, ADLS etc.• Experience in collaborating with business stakeholders to identify and meet data requirements• Experience in using Azure services and tools to ingest, egress, and transform data from...


  • india [24]7.ai Full time

    Job Role: Tech Lead – Data Engineering Job Location: Bangalore Working: Hybrid Summary of Essential Job Functions Design, develop, and maintain scalable and efficient data pipelines using Python, Pandas, Spark, and Google Dataflow. Strong background in relational databases such as SQL Server and MySQL along with a deep understanding of the Google...

  • Lead Data Engineer

    3 weeks ago


    india EXL Full time

    Location- Gurgaon/ Bangalore Experience- 7Yrs-8 Yrs Job Summary Looking for a versatile professional to join our team in the role of Hybrid Data Engineer and Database Administrator. This position requires a blend of expertise in Data Engineering and Database Administration, with a strong foundation in Oracle DBA roles and a demonstrated transition into data...


  • india TalentPulse Consultant Full time

    Lead ML Engineer Job Description : We are seeking a dynamic Lead ML Engineer to join our team in a remote capacity, operating within the UK shift timings of 2pm to 11pm IST. As the Lead ML Engineer, you will be responsible for leading the development and implementation of machine learning solutions to address complex business challenges.Responsibilities :-...

  • Data Engineer

    3 weeks ago


    india Joveo Full time

    We are looking for talented Data Engineers who are passionate about data engineering and have strong experience in building scalable data pipelines. As a Data Engineer at Joveo, you will play a leading role in designing and building datasets that power analytics for teams across Joveo. You will also be working closely with Data Analysts, Data Scientists, and...

  • Azure Data Engineer

    2 weeks ago


    india ValueLabs Full time

    Job Title: Azure Data Engineer Experience: 5+ years Must have Skills: ADF, SQL Server, Databricks Notebooks, Data Pipelines, SPROCs (Stored Procedures), DataWarehouse, Databricks DataLake Job Description: The Data Engineer is responsible for designing, developing, and maintaining data pipelines, data warehouses, and data lakes for an organization. They...

  • Lead Data Analyst

    1 month ago


    India CES Full time

    Job Role: We are looking for a Lead Data Engineer to join our team. Ideal Candidate: Overall 7+ years of experience. 5+ years of experience in Data Engineering / Modelling and SQL. 2+ years of experience in in Azure. Design, develop, and maintain data pipelines for ingestion, transformation, and loading of data into Azure Synapse Analytics. Transform...

  • Lead Data Analyst

    1 month ago


    India CES Full time

    Job Role: We are looking for a Lead Data Engineer to join our team.Ideal Candidate: Overall 7+ years of experience.5+ years of experience in Data Engineering / Modelling and SQL.2+ years of experience in in Azure. Design, develop, and maintain data pipelines for ingestion, transformation, and loading of data into Azure Synapse Analytics.Transform and clean...