Senior Data Engineer

4 days ago


Delhi Division, India MERNPLUS Full time

Job Description :

- Design, develop, and implement scalable and robust data pipelines using Apache Spark on Databricks.

- Leverage your deep expertise in Databricks to optimize performance and ensure efficient data processing.

- Work closely with data scientists and analysts to understand data requirements and translate them into technical specifications.

- Develop and maintain data pipelines using Databricks notebooks and leverage the Databricks multi-hop pipeline architecture.

- Build and manage data pipelines that are reliable, maintainable, and fault-tolerant.

- Implement best practices for data quality, including data validation, cleansing, and transformation.

- Collaborate with DevOps engineers to implement CI/CD pipelines for production deployments.

- Integrate with various data sources (databases, APIs, etc.) to extract, transform, and load data into our data lake/warehouse.

- Monitor and troubleshoot data pipelines to ensure smooth operation and identify potential issues.

- Stay up-to-date on the latest advancements in big data technologies and Databricks platform features.

Qualifications :

- 10+ years of experience in data engineering, data platforms, and data analytics.

- Proven track record of delivering successful data engineering projects with a focus on big data processing.

- Completed Data Engineering Professional certification (e., Databricks Certified Data Engineer)

- Minimum 6-8+ projects delivered with hands-on experience in development on Databricks.

- Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with deep expertise in at least one.

- Deep experience with distributed computing with Apache Spark and knowledge of Spark runtime internals.

- Proficiency in Apache Spark Programming with Databricks, Data Engineering with Databricks, and Optimizing Apache Spark on Databricks.

- Experience in Spark and/or Hadoop, Flink, Presto, or other popular big data engines.

- Familiarity with Databricks multi-hop pipeline architecture.

- Familiarity with CI/CD for production deployments.

- Working knowledge of MLOps concepts.

- Current knowledge across the breadth of Databricks product and platform features.

- Familiarity with optimizations for performance and scalability of big data pipelines

(ref:hirist.tech)

  • Delhi Division, India Karyarth consultants Full time

    Job Title : Senior Data Science EngineerLocation : New Delhi, India (Primary Location)Job Description : Are you ready to leverage your data science expertise to make a real impact?We are looking for a Senior Data Science Engineer to join our team. In this role, you will play a pivotal part in designing, implementing, and optimizing data-driven solutions to...


  • Delhi Division, India Discoveries Quintessential Pvt Ltd Full time

    Position : Senior Data Science EngineerLocation : New DelhiExperience : 6 - 9 YearsAre you ready to leverage your expertise in data science to make a tangible impact on the planet?We are a fast-growing deep-tech company at the forefront of clean energy innovation, using cutting-edge AI, Machine Learning, IoT, and advanced climate modeling to enhance energy...


  • Delhi Division, India Discoveries Quintessential Full time

    Job Opportunity : Senior Data Science EngineerAre you a data science wizard ready to make an impact? Join a dynamic, international team driving innovation in clean energy! Location : New Delhi, IndiaWhat You'll Do :- Design & optimize data pipelines for massive datasets. - Develop & deploy cutting-edge ML models. - Collaborate with experts to build...


  • Delhi, India Stealth Startup Full time

    About the Role:We are seeking a skilled and passionateSeniorData Engineerto join our team. As a Senior Data Engineer, you will play a critical role in designing, building, and maintaining scalable, reliable data pipelines and architectures. Your work will empower our organization to make data-driven decisions and deliver impactful solutions.Key...

  • Practice Manager

    4 days ago


    Delhi Division, India Hiring Prism LLP Full time

    Position Overview :We are seeking a Practice Manager - Data Engineering to lead and expand our Data and AI business. This strategic role requires a technology leader with deep expertise in data engineering, hands-on experience with platforms like Microsoft Fabric, Databricks, and Snowflake, and a strong foundation in Azure cloud technologies. You will drive...


  • Delhi, Delhi, India Pacific Data Integrators Full time

    Pacific Data Integrators is seeking a skilled Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, implementing, and optimizing data pipelines that ensure seamless data movement across systems.Key Responsibilities:Data Pipeline Development & Management: Design, implement, and optimize data...

  • Data Platform Engineer

    21 hours ago


    Delhi, India OSD Data Services Full time

    Data Platform Engineer Location : Remote Type : Internship About Us At OSD Data, we’re redefining how businesses leverage their data. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation. With a focus on data lakehouse technologies, we’ve built a...


  • Delhi, India OSD Data Services Full time

    Data Platform Engineer Location: Remote Type: Internship About Us At OSD Data, we’re redefining how businesses leverage their data. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation. With a focus on data lakehouse technologies, we’ve built...


  • Delhi, India OSD Data Services Full time

    Data Platform EngineerLocation : RemoteType : InternshipAbout UsAt OSD Data, we’re redefining how businesses leverage their data. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation. With a focus on data lakehouse technologies, we’ve built a...


  • Delhi, India OSD Data Services Full time

    Data Platform EngineerLocation : RemoteType : InternshipAbout UsAt OSD Data, we’re redefining how businesses leverage their data. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation. With a focus on data lakehouse technologies, we’ve built a...

  • Data Engineer

    4 days ago


    Delhi Division, India INTERNATIONAL ASSET RECONSTRUCTION COMPANY Full time

    Role : Data Engineer Job Description :We are looking for a Data Engineer to join our Data Science team. As a Data Engineer, you will play a key role in designing, implementing and maintaining the data infrastructure of our organization. Your primary responsibility will be to develop and optimize data pipelines, databases, and ETL processes, ensuring the...


  • Delhi, India OSD Data Services Full time

    Data Platform Engineer Location: Remote Type: Internship About Us At OSD Data, we’re redefining how businesses leverage their data. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation. With a focus on data lakehouse technologies, we’ve built a...


  • Delhi, Delhi, India Synchrony Full time

    About the RoleThis is an exciting opportunity for a seasoned Senior Data Engineering Lead to join our company's digital transformation journey. You will lead a team of data engineers and work closely with technology and functional teams to recommend data structures based on provisioning and consumption needs within each domain.The successful candidate will...


  • Delhi, India Edstem Technologies Full time

    Role Overview Description- The Senior Data Engineer will work collaboratively with Data Science and Analytics teams to build scalable, robust, and efficient data pipelines to enable data-driven decision making.- Responsible for designing data architectures, building/maintaining data pipelines, and integrating and transforming data from various sources to...


  • Delhi, India Edstem Technologies Full time

    Role Overview DescriptionThe Senior Data Engineer will work collaboratively with Data Science and Analytics teams to build scalable, robust, and efficient data pipelines to enable data-driven decision making.Responsible for designing data architectures, building/maintaining data pipelines, and integrating and transforming data from various sources to...


  • Delhi, India Perficient Full time

    Title - Senior Azure Data Factory Engineer (ADF)Location - Hyderabad/Chennai/BangaloreRole - FulltimeNotice period - ImmediateNo: of positions - 02Interview Mode - Virtual followed by In-PersonRequired Technical Expertise:Overall 6-10 Years of experience.Minimum 4+ years in Azure Data Factory development.Strong fundamental skills with SQL and...


  • Delhi Division, India Karyarth consultants Full time

    We are a cutting-edge deep-tech company driving innovation in the clean energy sector. Our mission is to accelerate the adoption of sustainable energy solutions by leveraging advanced technologies such as AI, ML, IoT, and climate modeling. We're looking for a Senior Software Engineer to design scalable cloud applications, optimize backend systems, and...


  • Delhi Division, India Perfios Software Full time

    Job Description :What your average day would look like :- Collaborate with product and engineering teams to understand requirements and devise possible solutions.- Explore existing research papers, ideas and codebases that can be leveraged in current tasks.- Search for open source datasets and/or design synthetic data pipelines (including data...


  • Delhi, Delhi, India Mastech Digital Full time

    Senior StreamSets Developer/EngineerKey Responsibilities:Design and develop data pipelines using StreamSets to meet business requirements.Monitor pipeline performance metrics, troubleshoot issues related to pipeline failures or data inconsistencies, and optimize pipelines for efficiency and scalability.Ensure data handling practices within StreamSets comply...


  • Delhi Division, India Presto InfoSolutions Full time

    Job Role : Senior Business Intelligence DeveloperCompany : Presto InfosolutionsJob Role Summary : We are seeking a highly skilled and experienced Senior Business Intelligence Developer to join our team. The ideal candidate will possess strong expertise in SAS-based BI development and reporting, with a proven track record of creating impactful dashboards,...