Data Engineering

4 weeks ago

delhi in VSHR UNNATI LLP Full time

SALARY : 10LPA - 16LPA


Job Overview :

Our client is seeking an experienced and detail-oriented Data Engineering and Machine Learning Tester to join our Quality Assurance team. The ideal candidate will have strong expertise in testing data pipelines, machine learning models, and large-scale data processing systems, specifically within the Databricks environment. This role is critical to ensuring the quality, accuracy, and reliability of our data engineering workflows and machine learning solutions.

Key Responsibilities :

Test Planning and Strategy :

- Develop, document, and maintain comprehensive test plans, test cases, and test scripts for data engineering and machine learning workflows.

- Collaborate with data engineers, data scientists, and product managers to understand requirements and define testing strategies tailored to the Databricks environment.

Data Pipeline Testing :

- Validate data ingestion, transformation, and processing workflows within Databricks.

- Perform end-to-end testing of ETL/ELT processes to ensure data accuracy, consistency, and performance.

- Verify data integrity across different stages of the pipeline and ensure that data meets business requirements.

Machine Learning Model Testing :

- Test and validate machine learning models developed on Databricks, ensuring they meet accuracy, performance, and reliability standards.

- Develop and execute tests for model training, validation, and deployment pipelines.

- Evaluate the performance of models on large datasets and ensure that they generalize well to unseen data


Automation and Continuous Integration :

- Implement automated testing solutions for data pipelines and machine learning models using tools compatible with Databricks.

- Integrate automated testing processes into the CI/CD pipeline to ensure continuous quality checks. Azure DevOps experience is a plus.

- Leverage Databricks notebooks and workflows for test automation where applicable.

Performance Testing :

- Conduct performance testing of data pipelines and machine learning models to identify bottlenecks and optimize processing time.

- Test the scalability of data engineering solutions to handle large volumes of data efficiently.

Collaboration and Communication :

- Work closely with cross-functional teams, including data engineers, data scientists, and DevOps, to resolve technical challenges and ensure the quality of data solutions.

- Provide clear and concise feedback on test results and collaborate with teams to address issues.

- Participate in sprint reviews and retrospectives, providing insights on test outcomes and quality metrics.

Qualifications :

Technical Skills :

- Strong experience with Databricks and related tools (Apache Spark, Delta Lake, MLflow).

- Proficiency in Python, SQL, and testing frameworks relevant to data engineering and machine learning.

- Experience with big data technologies on Azure platform.

- Knowledge of testing automation tools and CI/CD pipelines.

- Understanding of data governance, data quality, and data privacy practices.

Experience :

- 3+ years of experience in testing data engineering workflows and machine learning models, with a focus on big data environments.

- Proven experience working within Databricks or similar platforms.

- Prior experience in an Agile development environment is preferred.

Soft Skills :

- Excellent analytical and problem-solving skills.

- Strong communication skills, both verbal and written.

- Ability to work independently and as part of a collaborative team.

Benefits :

- Opportunities for professional growth and development.

- Collaborative and innovative work environment.

(ref:hirist.tech)

  • Data engineer

    4 days ago

    New Delhi, India Tatras Data Full time

    Job Title: Data Engineer - Data Fabric/BricksLocation: RemoteJob Summary:We are seeking an experienced Data Engineer to join our team as contractor. The ideal candidate will have a strong background in designing, implementing, and managing large-scale data platforms using Data Fabric or Data Bricks. The candidate should have experience working with both...

  • Data Platform Engineer

    3 weeks ago

    Delhi, Delhi, India OSD Data Services Full time

    Data Platform EngineerLocation : RemoteType : InternshipAbout UsAt OSD Data, we're redefining how businesses leverage their data. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation. With a focus on data lakehouse technologies, we've built a...

  • Data Engineer

    4 weeks ago

    delhi, in Nacerz Global Full time

    Our client enable its clients to thrive in Next and to create business value through expert strategies, customer-centric experience design, and world-class product engineering. The future of business is disruptive, transformative and becoming digital to the core.In our 20 + years in IT, never before have we seen such a dire need for transformation in every...

  • Data Engineer

    4 weeks ago

    delhi, in MindTrilogy IT Solutions Pvt. Ltd Full time

    Job Title : Data Engineer Experience Required : 57 years Job Description & Responsibilities : - Support the design, development, and optimization of data processing workflows and pipelines tailored for NVIDIA GPU infrastructure.- Work closely with system engineers to efficiently execute AI jobs, ensuring optimal performance on NVIDIA platforms.- Compare and...

  • Data Platform Engineer

    3 weeks ago

    New Delhi, India OSD Data Services Full time

    Data Platform EngineerLocation : RemoteType : InternshipAbout UsAt OSD Data, we’re redefining how businesses leverage their data. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation. With a focus on data lakehouse technologies, we’ve built a...

  • Data Platform Engineer

    3 weeks ago

    New Delhi, India OSD Data Services Full time

    Data Platform EngineerLocation : RemoteType : InternshipAbout UsAt OSD Data, we’re redefining how businesses leverage their data. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation. With a focus on data lakehouse technologies, we’ve built a...

  • Senior Data Engineer

    4 weeks ago

    West Delhi, India Wavicle Data Solutions Full time

    Job Description:We are seeking a highly experienced Senior Data Engineer with over 5 years of expertise in data engineering.As a Senior Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS/Azure, Databricks/Snowflake, SQL, and leadership skills will be crucial for success.Key...

  • Azure Data Engineer

    4 weeks ago

    delhi, in Breezy Talent Full time

    We are hiring for our client Talpro India Private Limited. Azure Data Engineer : We are looking for a skilled and experienced Azure Data Engineer with expertise in ADF (Azure Data Factory), strong RDBMS and SQL skills, and familiarity with Informatica PWC and Oracle PL/SQL. As an Azure Data Engineer, you will be responsible for understanding and converting...

  • Data Engineer

    4 weeks ago

    delhi, in Innovistors Technological Solutions Pvt Ltd Full time

    Job Description :As part of the team, you will be responsible for building and running the data pipelines and services that are required to support business functions/reports/dashboard.. We are heavily dependent on BigQuery/Snowflake, Airflow, Stitch/Fivetran, dbt , Tableau/Looker for our business intelligence and embrace AWS with some GCP.As a Data Engineer...

  • Lead Data Engineer

    2 weeks ago

    Delhi, Delhi, India Pacific Data Integrators Full time

    Job Title:Lead Data Engineer (IDMC-CDI)Location:RemoteMode of Work: Work from OfficeWorking Hours: US Shift (5 PM to 2 AM IST)Primary Skills: SQL, IDMC (CDI), ETL ConceptsExperience: 8+ YearsAbout the JobAre you someone with a deep understanding of ETL and a strong background in developing IDMC ETL-based solutions, capable of developing, documenting, unit...

  • Aptus Data Labs

    4 weeks ago

    delhi, in Aptus Data Labs Full time

    Role - Lead/Sr AI EngineerLocation - Bangalore (Currently WFH)Exp - 4- 7 yrsNotice - Candidates with less notice period will be : - Designing and implementing AI and machine learning models to solve business problems - Collaborating with cross-functional teams to understand requirements and deliver tailored AI solutions - Conducting research and prototyping...

  • Senior Data Engineer

    4 weeks ago

    delhi, in Uncap Research Labs Full time

    Senior Data EngineerOUR STORY : The tech space is crowded, but most solutions feel like they're cut from the same cloth-missing the mark on what businesses truly need. Organizations are chasing innovation, but too often, it comes at the expense of flexibility, independence, and real technological depthAt AT DAWN Technologies, we're a niche startup...

  • Junior Data Engineer

    3 weeks ago

    delhi, in Think Future Tech Full time

    About the Role :We are seeking a highly motivated and talented Junior Data Engineer to join our growing data team. In this role, you will play a crucial part in building and maintaining our data infrastructure, enabling data-driven decisions across the organization. You will work closely with senior engineers and data scientists, contributing to the...

  • Data Engineer

    4 weeks ago

    delhi, in HireXtra Full time

    About the job :We are seeking a skilled AWS Data Engineer with expertise in various AWS services. The an ideal candidate will have hands-on experience with Lambda, Glue, SNS, SQS, Step Functions, PySpark, Python, Athena, CloudWatch, S3, and more. The successful candidate should also have working experience with various data file formats such as JSON, XML,...

  • Data Engineer

    3 weeks ago

    delhi, in Move Forward Technologies Pvt. Ltd. Full time

    Hiring: Data Engineer - SSIS & Tableau (Remote)MoveForward Technologies is looking for a highly skilled Data Engineer with expertise in SSIS, Tableau, SQL, ETL, and Data Modeling to join our team. If you have 5-10 years of experience in data engineering, love working with data pipelines, and thrive in an Agile environment, we'd love to hear from you!...

  • Lead Data Engineer

    4 weeks ago

    delhi, in TwinPacs Sdn Bhd Full time

    Seeking for a Lead data Engineer with a wealth of knowledge in managing legacy SQL Server installations and creating ETL procedures with PySpark Notebooks and Microsoft Fabric. The ideal applicant will have a solid foundation in Spark-based development, exhibit a high level of SQL competence, and feel at ease working both independently and in teams, as well...

  • Data Quality Engineer

    4 weeks ago

    delhi, in Coders Brain Technology Private Limited Full time

    Job Title : Data Quality (IDQ) EngineerLocation : RemoteExperience : 6+ YearsNotice Period : ImmediateJob Type : Full-timeJob Overview :We are seeking experienced Informatica Data Quality (IDQ) Engineers who have hands-on expertise in data discovery, standardization, and executing data quality rules. As a Data Quality Engineer, you will work closely with...

  • Data Engineer

    4 weeks ago

    delhi, in FANISKO PRIVATE LIMITED Full time

    Job Title : Data Engineer with AIML ExpertiseYears of Experience : 3+ yearsQualifications : Any UG/PGLocation : RemoteEmployment Type : Contract / Full-Time Employment (FTE)Required by Date : Immediate startShare your profiles to hr@fanisko.comJob Description : - Data Pipeline Development : Design, implement, and maintain scalable and efficient data...

  • Senior Data Warehousing Engineer

    4 weeks ago

    delhi, in Hire Alpha Full time

    About the Role : We are seeking a highly skilled and motivated Senior Data Warehouse Engineer to join our growing team. In this critical role, you will play a key part in designing, developing, and maintaining our data warehouse infrastructure. You will be responsible for building and optimizing data pipelines, ensuring data quality and integrity, and...

  • Data Engineer

    4 weeks ago

    delhi, in Yo HR Consultancy Full time

    Title: Data Engineer (Technical Lead)Experience : 8 years to 14 yearsWork Mode: RemoteJob Description : As part of the team, you will be responsible for building and running the data pipelines and services that are required to support business functions/reports/dashboard.. We are heavily dependent on BigQuery/Snowflake, Airflow, Stitch/Fivetran, dbt ,...