Senior Big Data Engineer

4 days ago

Gurgaon, Haryana, India Pylon Management Consulting Full time

About the Role :

We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team. In this role, you will be responsible for designing, developing, and maintaining our data infrastructure and pipelines. You will work with large datasets, implement scalable solutions, and ensure data quality and reliability. The ideal candidate will have a strong background in big data technologies, a passion for data, and the ability to drive innovation.

Responsibilities :

Design and Development :

- Design and implement scalable and robust data pipelines for data ingestion, processing, and storage.

- Develop and maintain data warehouses and data lakes using cloud-based and on-premise technologies.

- Build and optimize ETL/ELT processes to ensure efficient data flow.

- Develop and implement data quality checks and monitoring systems.

Infrastructure Management :

- Manage and optimize big data infrastructure, including clusters and storage systems.

- Ensure high availability and performance of data platforms.

- Implement and maintain data security and access control policies.

Performance Optimization :

- Identify and resolve performance bottlenecks in data pipelines and systems.

- Optimize queries and data structures for efficient data retrieval.

- Monitor and analyze system performance to ensure optimal resource utilization.

Collaboration and Communication :

- Collaborate with data scientists, analysts, and other engineers to understand data requirements and deliver solutions.

- Communicate effectively with stakeholders to provide updates and insights on data projects.

- Document technical designs, processes, and best practices.

Innovation and Research :

- Stay up-to-date with the latest big data technologies and trends.

- Evaluate and recommend new tools and technologies to improve data infrastructure and processes.

- Contribute to the development of data engineering best practices.

Required Technical Skills :

Big Data Technologies :

- Apache Hadoop (HDFS, MapReduce, YARN)

- Apache Spark (Spark Core, Spark SQL, Spark Streaming)

- Apache Kafka

- Apache Hive, Apache Impala

Cloud platforms :

- AWS (EMR, S3, Glue, Redshift), Google Cloud Platform (Dataproc, BigQuery, Dataflow), or Azure (HDInsight, Data Lake Storage, Azure Synapse Analytics)

Programming Languages :

- Python (Pandas, NumPy, PySpark)

- Scala

- SQL

Data Warehousing and Databases :

- Data warehousing concepts and principles

- Relational databases (e.g., PostgreSQL, MySQL)

- NoSQL databases (e.g., Cassandra, MongoDB)

ETL/ELT Tools :

- Apache Airflow, Luigi, or similar orchestration tools.

Version Control :

- Git

Operating Systems :

- Linux/Unix

Preferred Skills :

- Experience with containerization (Docker, Kubernetes).

- Experience with data visualization tools (e.g., Tableau, Power BI).

- Experience with machine learning pipelines.

- Understanding of data governance and compliance.

- Experience with streaming data processing.

Qualifications :

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

- 5-8 years of experience1 in big data engineering.

- Proven experience in designing and implementing large-scale data solutions.

- Strong problem-solving and analytical skills.

- Excellent communication and collaboration skills.

(ref:hirist.tech)

  • Senior Big Data Engineer

    3 hours ago

    Gurgaon, Haryana, India MakeMyTrip Full time

    Job DescriptionAs a Senior Big Data Engineer at MakeMyTrip, you will be responsible for designing, building, and maintaining large-scale data architectures and systems. You will work closely with cross-functional teams to develop and implement data-driven solutions that drive business growth and customer satisfaction.Your key responsibilities will include1....

  • Big Data Engineer

    7 days ago

    Gurgaon, Haryana, India Coders Brain Technology Private Limited Full time

    Company OverviewCoders Brain Technology Private Limited is a leading technology firm dedicated to harnessing the power of data. Job Description We are seeking an experienced Technical Lead with expertise in Big Data engineering initiatives. As part of our team, you will lead and manage projects ensuring robust and scalable solutions. Your responsibilities...

  • Data Quality Engineer

    17 hours ago

    Gurgaon, Haryana, India Summit Consulting Services Full time

    Job SummaryWe are seeking a highly skilled Data Quality Engineer to join our team at Summit Consulting Services. As a key member of our data engineering team, you will be responsible for leading quality engineering efforts and developing robust test strategies to ensure the scalability, reliability, and performance of our data systems.About the RoleThe...

  • Big Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India Chiselon Technologies Pvt Ltd Full time

    # Hiring Alert #We are looking for Immediate Joiners Skills:Big Data with GCP GCP; Hive; PySpark; Python GCP Data Engineer with hands-on experience of 3+ years in Big Query, Data Proc, Airflow, Cloud Composer, GCP Hydra Services, Cloud Data Optimization "Join us and become an integral part of our success story. Apply today and let's achieve great things...

  • Senior Data Engineer

    7 days ago

    Gurgaon, Haryana, India Invokhr Full time

    Position: Senior Data Engineer / Data Engineer Desired Experience: 3-8 years Salary: Best-in-industry You will act as a key member of the Data consulting team, working directly with the partners and senior stakeholders of the clients designing and implementing big data and analytics solutions. Communication and organisation skills are keys for this position,...

  • Big Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India dunnhumby Full time

    We're looking for a Big Data Engineer who expects more from their career. It's a chance to extend and improve dunnhumby's Data Engineering Team It's an opportunity to work with a market-leading business to explore new opportunities for us and influence global retailers. Joining our team, you'll work with world class and passionate people which is part...

  • Senior Big Data Engineer

    2 days ago

    Gurgaon, Haryana, India Alp Consulting Ltd. Full time

    About the RoleThis Senior Data Engineer position is highly dynamic, with an emphasis on automating data processes and creating reusable, scalable solutions in a fast-paced environment.Key ResponsibilitiesDesign and implement scalable data engineering solutions.Manage and optimize data workflows from development to production.Take ownership of the development...

  • Big Data Developer

    3 weeks ago

    Gurgaon, Haryana, India Coforge Full time

    Please share resume at nishika.arora@coforge.com with below details: Experience- CTC- Expected CTC- Notice Period- Current Location- Preferred Location- Job Details SKILLS - BIG Data Engineer Mandatory Skills- Big Data, Hadoop, Hive, Spark , Scala , Airflow (good to have) Experience - 5+ years Job Location - Gurugram Detailed Job Description- Bachelor's or...

  • Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India Gravity Infosolutions Full time

    Job Title: Senior Data Engineer (Contexa, AWS, Spark) Location : Gurugram, India (Hybrid – 3 days in office, 2 days WFH) Term : 12+ months Job Overview: We are seeking a Senior Data Engineer with expertise in VMware Contexa, Spark, AWS, and ETL for a high-impact role in the banking & finance sector . The ideal candidate will have strong experience in big...

  • Senior Data Engineer

    3 weeks ago

    Gurgaon, Haryana, India PAR Technology Full time

    Hi there Have you noticed that the restaurant industry is changing?We are PAR and our purpose is:To deliver solutions that connect people to the restaurants, meals, and moments they love.At PAR Technology, our relentless drive for innovation and unwavering commitment to customer success are at the heart of everything we do. We lead the restaurant and retail...

  • Senior Data Engineer

    1 week ago

    Gurgaon, Haryana, India PAR Technology Full time

    Hi there Have you noticed that the restaurant industry is changing? We are PAR and our purpose is: To deliver solutions that connect people to the restaurants, meals, and moments they love. At PAR Technology, our relentless drive for innovation and unwavering commitment to customer success are at the heart of everything we do. We lead the restaurant and...

  • Big Data Engineer, Global Finance Technology

    1 week ago

    Gurgaon, Haryana, India ADCI - Karnataka Full time

    Are you an experienced Big Data Engineer passionate about building scalable and enterprise-level data engineering solutions? This opportunity is a great fit for an experienced Big Data Engineer who enjoys working on large-scale computing leveraging AWS distributed systems. In addition to technical expertise, you will invest time to understand the needs of...

  • Big Data Developer

    4 weeks ago

    Gurgaon, Haryana, India Talentonlease Pvt ltd Full time

    Key Responsibilities :Data System Development & Optimization :- Design, develop, and implement efficient data pipelines and data storage solutions using Big Data technologies.- Build robust, scalable, and high-performance data processing systems using distributed frameworks such as Apache Spark (Core, Streaming, SQL), Flink, or Storm.- Optimize and...

  • Big Data Developer

    3 weeks ago

    Gurgaon, Haryana, India Talentonlease Pvt ltd Full time

    Key Responsibilities :Data System Development & Optimization :- Design, develop, and implement efficient data pipelines and data storage solutions using Big Data technologies.- Build robust, scalable, and high-performance data processing systems using distributed frameworks such as Apache Spark (Core, Streaming, SQL), Flink, or Storm.- Optimize and...

  • Big Data Engineer, Global Finance Technology

    1 week ago

    Gurgaon, Haryana, India Amazon Full time

    Are you an experienced Big Data Engineer passionate about building scalable and enterprise-level data engineering solutions? This opportunity is a great fit for an experienced Big Data Engineer who enjoys working on large-scale computing leveraging AWS distributed systems. In addition to technical expertise, you will invest time to understand the needs of...

  • Senior Data Engineer

    3 weeks ago

    Gurgaon, Haryana, India PAR Technology Full time

    Hi there Have you noticed that the restaurant industry is changing?We are PAR and our purpose is:To deliver solutions that connect people to the restaurants, meals, and moments they love.At PAR Technology, our relentless drive for innovation and unwavering commitment to customer success are at the heart of everything we do. We lead the restaurant and retail...

  • Senior Data Engineer

    1 week ago

    Gurgaon, Haryana, India PAR Technology Full time

    Hi there Have you noticed that the restaurant industry is changing?We are PAR and our purpose is:To deliver solutions that connect people to the restaurants, meals, and moments they love.At PAR Technology, our relentless drive for innovation and unwavering commitment to customer success are at the heart of everything we do. We lead the restaurant and retail...

  • Senior Data Engineer

    7 days ago

    Gurgaon, Haryana, India Recruitment Agency Full time

    Company OverviewWe are a leading Recruitment Agency seeking an experienced Senior Data Engineer to lead our data architecture initiatives. As a key member of our team, you will collaborate with architects, developers, and engineers to design and develop large-scale data platforms.Job DescriptionWe are looking for a skilled Cloud Data Architect Lead to join...

  • Technical Lead

    7 days ago

    Gurgaon, Haryana, India Coders Brain Technology Private Limited Full time

    Coders Brain Technology Private Limited seeks a skilled Technical Lead with expertise in Big Data engineering initiatives to join our dynamic team. As a seasoned professional, you will leverage your experience to lead and manage big data engineering projects, ensuring robust and scalable solutions that meet our business requirements. About the Role This...

  • Big Data Analytics Expert

    6 days ago

    Gurgaon, Haryana, India Pluang Technologies Full time

    ResponsibilitiesAs a Senior Data Platform Specialist at Pluang Technologies, you will be responsible for designing, building, and scaling our data infrastructure to support real-time analytics, big data processing, and machine learning deployments.You will create near-real-time ETL pipelines, ensure data consistency across our Data Lake and Warehouse, and...