Big Data Automation Engineer

17 hours ago


Delhi, Delhi, India PubMatic Full time
Job Description

We are seeking a highly skilled Big Data Automation Engineer to join our team at PubMatic. As a key member of our engineering team, you will be responsible for designing, developing, and maintaining automation frameworks for big data workflows.

Responsibilities:
  • Designing and implementing automation frameworks using Python to automate big data workflows such as ingestion, aggregation, and ETL processing.
  • Testing big data ingestion and aggregation flows using Spark shell and related queries.
  • Debugging and troubleshooting issues within the big data ecosystem.
  • Setting up the Big Data platform and Hadoop ecosystem for testing.

Requirements

To succeed in this role, you should have:
  • Good hands-on experience with Unix/Linux.
  • Good hands-on experience in writing Python codes.
  • Understanding of QA methodologies.
  • Experience in Big Data platform & data analytics testing is an advantage.
  • Knowledge on distributed systems and technologies like Hadoop and Spark.

Qualifications

The ideal candidate should have 1-3 years of experience working as SDET and doing meaningful automation. You should have good programming skills, with Python being preferred. Additionally, you should have hands-on experience in automating backend applications (e.g., database, REST APIs) and knowledge of relational databases and SQL. Good debugging skills, working experience in Linux/Unix environment, and good understanding of testing methodologies are also essential. Quick learner, good team member with positive attitude, and good verbal and written communication skills are also required.

Salary

The estimated salary for this position is $120,000 - $180,000 per year, depending on location and experience.
  • Big Data Engineer

    2 weeks ago


    Delhi, Delhi, India Wipro Full time

    We are seeking a highly skilled Big Data Engineer to join our team at Wipro. As a Big Data Engineer, you will play a key role in designing, developing, and maintaining large-scale data processing systems.The ideal candidate will have a strong background in Hadoop and related technologies such as Scala. You will work closely with cross-functional teams to...


  • Delhi, Delhi, India LTIMindtree Full time

    We are seeking an experienced Data Engineer to join our team at LTIMindtree. As a key member of our big data solutions group, you will be responsible for designing and developing scalable data platforms using modern technologies like Spark, HBase, Hive, and other Hadoop ecosystem tools.About the RoleIn this role, you will work closely with cross-functional...

  • Big Data Engineer

    1 month ago


    Delhi, Delhi, India Talent Pro Full time

    Hadoop Developer Job DescriptionWe are currently looking for a skilled and experienced Hadoop Developer to join our team at Talent Pro.As a Hadoop Developer, you will be responsible for designing, developing, and maintaining large-scale big data processing pipelines using the Hadoop ecosystem (HDFS, MapReduce) and Spark.The ideal candidate will have a strong...


  • Delhi, Delhi, India HCLTech Full time

    About HCLTechWe are a leading global technology company with a strong presence in the IT services industry. Our team of experts is passionate about delivering innovative solutions that drive business growth and success.Job Title: Senior Big Data Engineer - ETL Framework DeveloperLocation: Pan IndiaSalary: ₹ 25,00,000 - ₹ 40,00,000 per annumJob...

  • Engineering Manager

    3 weeks ago


    Delhi, Delhi, India PROTEGRITY INDIA PRIVATE LIMITED Full time

    About the RoleProtegrity - Engineering Manager - Big DataProtegrity is a pioneering company in data protection, and we're seeking a seasoned Engineering Manager to lead our Big Data team. This role requires a strategic thinker with expertise in data security and a passion for mentoring and guiding technical teams.Key ResponsibilitiesManage the research and...


  • Delhi, Delhi, India Freelancer Recruiter Full time

    Job RequirementsWe are seeking an experienced Big Data Infrastructure Specialist to manage and maintain complex Big Data application stacks. The ideal candidate will have a deep understanding of HDFS, YARN, Spark, Hive, and HBase, as well as experience with infrastructure configurations, high availability, and resource allocation.Key Responsibilities:Design...

  • Cloud Data Engineer

    17 hours ago


    Delhi, Delhi, India Tata Consultancy Services Full time

    Job Role: Cloud Data Engineer - Big Data Solutions SpecialistAbout the Role: Tata Consultancy Services is seeking a skilled Cloud Data Engineer to join our team in Chennai. As a key member of our big data solutions team, you will be responsible for developing and deploying scalable big data pipelines on Google Cloud Platform (GCP).Responsibilities: Develop...

  • Senior Data Engineer

    2 weeks ago


    Delhi, Delhi, India PRONEXUS CONSULTING PRIVATE LIMITED Full time

    **Job Title:** Senior Data Engineer - Big Data Pipeline SpecialistAt Pronexus Consulting Private Limited, we are seeking an experienced and highly skilled Senior Data Engineer - Big Data Pipeline Specialist to join our team. This role will involve designing, developing, and implementing large-scale data pipelines using various technologies such as Spark,...


  • Delhi, Delhi, India NEXUS CONSULTING SOLUTIONS Full time

    We are seeking a skilled Data Engineer to join our team at NEXUS CONSULTING SOLUTIONS. This role requires experience in working with large data sets and a strong understanding of big data systems.Job DescriptionA successful candidate will have hands-on experience in Python and Scala programming languages. Additionally, they should be familiar with the Spark...


  • Delhi, Delhi, India NucleusTeq Consulting Pvt. Ltd. Full time

    About the RoleAs a Big Data Solutions Architect, you will be responsible for finalizing the scope of the system and delivering Big Data solutions. This position requires in-depth knowledge of Hadoop, Spark, and similar frameworks.Responsibilities- Lead a team of engineers and have a good attitude toward learning the domain and implementation.- Collaborate...


  • Delhi, Delhi, India Emperen Technologies Full time

    About the Role">We are seeking a highly skilled Big Data Quality Assurance Specialist to join our team at Emperen Technologies. This is an exciting opportunity for a motivated individual to work on cutting-edge projects and contribute to the development of high-quality data solutions.Key Responsibilities">The successful candidate will be responsible for...


  • Delhi, Delhi, India Luxoft Full time

    Transforming Business with Data Engineering ExpertiseLuxoft is seeking a highly skilled Senior Data Architect to lead the development of innovative big data solutions. As a key member of our Data Engineering & Analytics team, you will design and implement scalable data architecture and pipelines, leveraging your expertise in Python/Scala, Spark, SQL, and...


  • Delhi, Delhi, India Tata Consultancy Services Full time

    Tata Consultancy Services is seeking a highly skilled Data Engineer to join our team in building enterprise big data solutions. With over 10 years of experience, you will play a critical role in designing and implementing large-scale data processing systems using Apache Hadoop components and Spark.As a Data Engineer at TCS, you will be responsible for...

  • Cloud Data Engineer

    17 hours ago


    Delhi, Delhi, India Tata Consultancy Services Full time

    About the RoleWe are seeking an experienced Cloud Data Engineer to join our team at Tata Consultancy Services. As a key member of our data engineering team, you will play a vital role in designing, building, and deploying big data pipelines in Google Cloud Platform (GCP).Key ResponsibilitiesDesign and develop scalable big data architectures on GCP using...

  • Cloud Data Engineer

    1 week ago


    Delhi, Delhi, India Tata Consultancy Services Full time

    Tata Consultancy Services is seeking a highly skilled Cloud Data Engineer - Big Data Analytics to join our team. We are conducting interviews for this role, and we encourage qualified candidates to apply. Company Overview


  • Delhi, Delhi, India Freelancer Recruiter Full time

    At Freelancer Recruiter, we're looking for a highly skilled Big Data DevOps Engineer to join our team. If you have 5+ years of experience managing the Big Data application stack, including HDFS, YARN, Spark, Hive, and Hbase, this could be the perfect opportunity for you.We're offering an estimated salary range of $120,000 - $180,000 per year, depending on...


  • Delhi, Delhi, India CodeVyasa Full time

    Job DescriptionWe are seeking a highly skilled Senior Big Data Engineer to join our team in Bangalore. As a key member of our data engineering team, you will be responsible for designing, implementing, and maintaining large-scale data processing pipelines using PySpark for both streaming and batch processing.Key Responsibilities:- Design, implement, and...

  • Data Engineering Lead

    2 weeks ago


    Delhi, Delhi, India Lancetech solutions Full time

    **About Lancetech Solutions**We are a cutting-edge technology company that provides innovative solutions to meet the ever-evolving needs of our clients. As a data engineering lead, you will play a vital role in shaping the future of our data-driven products.**Salary and Benefits**We offer a competitive salary range of $180,000 - $250,000 per annum, depending...

  • Big Data Engineer

    3 days ago


    Delhi, Delhi, India Talent Pro Full time

    Job Title: Big Data Engineer - Design and Develop Scalable Solutions">About the Role:Talent Pro is seeking a highly skilled Hadoop Developer to join our team. As a Big Data Engineer, you will design and develop large-scale big data processing pipelines using Hadoop ecosystem (HDFS, MapReduce) and Spark.Key Responsibilities:Design, develop, test, deploy, and...


  • Delhi, Delhi, India CAI Info India Pvt Ltd. Full time

    About the RoleCAI Info India Pvt Ltd. is seeking an experienced AWS Data Engineer to design and develop robust data lakes, leveraging cutting-edge technologies such as S3, Glue, Lake Formation, EMR, Kinesis, RDS, DMS, and Redshift.This full-time remote position requires a strong background in cloud computing, data engineering, and software development, with...