Big Data Engineer

1 week ago


Gurgaon, Haryana, India Intraedge Technologies Ltd. Full time

Job Description :
We are seeking a skilled Big Data GCP Engineer to join our team.

The ideal candidate will have substantial experience with Google Cloud Platform (GCP) and a strong background in Big Data technologies.

This role involves working with various GCP services and tools to design, develop, and optimize data processing pipelines and workflows.


Key Responsibilities :

GCP Data Engineering :

  • Design, implement, and maintain data processing pipelines using GCP services such as BigQuery, DataProc, Airflow, Cloud Composer, and GCP Hydra Services.
  • Optimize and manage data storage and retrieval processes to ensure efficient performance and costeffectiveness.

Big Data Technologies :

  • Utilize Hive, PySpark, and Python to develop and manage Big Data solutions.
  • Implement data transformations, aggregations, and data enrichment processes.

Data Pipeline Development :

  • Develop and maintain ETL processes to integrate and process large datasets from various sources.
  • Ensure data quality, reliability, and security in all data processing activities.

Workflow Automation :

  • Use Airflow and Cloud Composer to automate data workflows and orchestrate complex data processing tasks.
  • Monitor and troubleshoot data pipelines to ensure smooth and uninterrupted operations.

Performance Optimization:

  • Optimize data processing jobs for performance and scalability.
  • Implement best practices for data storage, indexing, and partitioning in BigQuery and other GCP services.

Collaboration and Support :

  • Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver effective solutions.
  • Provide technical support and troubleshooting for datarelated issues.

Documentation and Best Practices :

  • Maintain comprehensive documentation of data pipelines, workflows, and processes.
  • Adhere to industry best practices and standards in Big Data and cloud engineering.

Required Skills and Qualifications :

  • Education: Bachelor's degree in Computer Science, Information Technology, or a related field.

Experience :

  • 3+ years of handson experience with GCP services such as BigQuery, DataProc, Airflow, Cloud Composer, and GCP Hydra Services.
  • 38 years of experience in Big Data engineering.

Technical Skills :

  • Proficient in Hive, PySpark, and Python.
  • Strong understanding of data warehousing concepts and best practices.
  • Experience with cloud data optimization and cost management.

Soft Skills :

  • Strong problemsolving and analytical skills.
  • Excellent communication skills, both verbal and written.
  • Ability to work effectively in a teamoriented, collaborative environment.

Personal Attributes :

  • Selfmotivated, proactive, and able to work independently.
  • Strong attention to detail and commitment to delivering highquality solutions.
)
  • Big Data Engineer

    1 week ago


    Gurgaon, Haryana, India virtusa consulting services pvt. ltd. Full time

    Apply for Big Data Engineer, Career Progress Consultants in Gurgaon for 6 - 9 Year of Experience on

  • Big Data Engineer

    2 weeks ago


    Gurgaon, Haryana, India virtusa consulting services pvt. ltd. Full time

    Apply for Big Data Engineer, Career Progress Consultants in Gurgaon for 6 - 9 Year of Experience on

  • Big Data Developer

    1 week ago


    Gurgaon, Haryana, India Workassist Full time

    Our hiring partner is looking for :This is a full-time on-site role for a Big Data Developer at Big Four Consulting Firm Gurugram. The Big Data Developer will be responsible for tasks related to Data Engineering, Extract Transform Load (ETL), Big Data, Data Warehousing, and Software Data Engineering, Big Data, Extract Transform Load (ETL), and Data...

  • Big Data Engineer

    1 week ago


    Gurgaon, Haryana, India Chiselon Technologies Pvt Ltd Full time

    # Hiring Alert #We are looking for Immediate Joiners Skills:Big Data with GCP GCP; Hive; PySpark; Python GCP Data Engineer with hands-on experience of 3+ years in Big Query, Data Proc, Airflow, Cloud Composer, GCP Hydra Services, Cloud Data Optimization "Join us and become an integral part of our success story. Apply today and let's achieve great things...

  • Big Data Developer

    1 week ago


    Gurgaon, Haryana, India Rocket Consulting Full time

    Big Data EngineerPosition Overview :We are seeking a skilled Big Data Engineer to join our dynamic team. The ideal candidate will have expertise in designing, implementing, and maintaining large-scale data processing systems using cutting-edge technologies. As a Big Data Engineer, you will play a key role in building and optimizing data pipelines, data...

  • Big Data Engineer

    1 week ago


    Gurgaon, Haryana, India OmegaHires Full time

    Job Title:Big Data Engineer Duration: Full-time on OmegaHires Payroll Location: Gurugram (Should be located near NCR to attend office for 3 days a week) Should understand the Credit Card domain and good to have knowledge on Compliance & Operational Risk . Good analytics experience to do data discovery, analyze the gaps in data, partner with Data Owners,...


  • Gurgaon, Haryana, India Publicis Sapient Full time

    Job Description : As Manager, Data Engineering, you will be responsible for translating client requirements into design, architecting, and implementing Azure Cloud based big data solutions for clients. Your role will be focused on delivering high quality solutions by independently driving design discussions related to Data Ingestion, Transformation &...

  • Big Data Engineer

    1 week ago


    Gurgaon, Haryana, India dunnhumby Full time

    We're looking for a Big Data Engineer who expects more from their career. It's a chance to extend and improve dunnhumby's Data Engineering Team It's an opportunity to work with a market-leading business to explore new opportunities for us and influence global retailers. Joining our team, you'll work with world class and passionate people which is part of...

  • Big Data

    1 week ago


    Gurgaon, Haryana, India Virtusa Full time

    Big Data - Informatica Developer - CREQ188336 Description Must have 6-9 year of experience of create, design, and implement data processing jobs to transform the data.Demonstrate proficiency in writing shell script and SQL queries.Hands on Exp with Oracle and DB2 databases.Must have Big Data tech stack experience Hadoop, Hive, Mapr etc.Good Communication...


  • Gurgaon, Haryana, India Waytogo Consultants Full time

    Job Description :In this role, you will play a critical part in designing, developing, and maintaining big data processing pipelines using your expertise in Scala and Apache Spark. You will collaborate closely with data engineers and data scientists to understand their data needs and translate those requirements into robust technical...

  • Big Data Developer

    1 week ago


    Gurgaon, Haryana, India Shya Workforce Solutions Full time

    Collaborate with enterprise architects, data architects, developers & engineers, data scientists, andinformation designers to lead the identification and definition of required data structures, formats,pipelines, metadata, and workload orchestration capabilitiesExperience in service architecture, development, and high performance and scalability.Experience...


  • Gurgaon, Haryana, India BYD Full time

    【Working Location】Gurgaon, India【Working language】English & ChineseKnowing Chinese language would be preferred【Job Responsibilities】l Responsible for the architecture design of Google Cloud's big data and the application of new technologies on the platform, from both functional and non-functional perspectives.Participate in the design and...


  • Gurgaon, Haryana, India BYD Full time

    【Working Location】Gurgaon, India【Working language】English & ChineseKnowing Chinese language would be preferred【Job Responsibilities】l Responsible for the architecture design of Google Cloud's big data and the application of new technologies on the platform, from both functional and non-functional perspectives. Participate in the design and...

  • Big Data Engineer

    1 week ago


    Gurgaon, Haryana, India HuQuo Consulting Pvt. Ltd. Full time

    Big Data Engineer Must-Have : 5+ years of experience in Software and IT industry 3 + years of relevant work experience in data engineering Hands on in Python/Scala data engineering application development Hands on in developing streaming application. Knowledge on Test automation and scripting Experience with CI/CD process and tools (GIT, Jenkins, SonarQube)...

  • Big Data Developer

    1 week ago


    Gurgaon, Haryana, India Shya Workforce Solutions Full time

    Join our team to:Collaborate with enterprise architects, data architects, developers & engineers, data scientists, and information designers to lead the identification and definition of required data structures, formats, pipelines, metadata, and workload orchestration capabilitiesExperience in service architecture, development, and high performance and...

  • Big Data Developer

    1 week ago


    Gurgaon, Haryana, India Impetus Full time

    About ImpetusImpetus Technologies is a digital engineering company focused on delivering expert services and products to help enterprises achieve their transformation goals. We solve the analytics, AI, and cloud puzzle, enabling businesses to drive unmatched innovation and growth.Founded in 1991, we are cloud and data engineering leaders providing solutions...

  • Engineer- Big Data

    1 week ago


    Gurgaon, Haryana, India American Express Full time

    Welcome to American Express - where you can lead the way with our backing!Joining the team at American Express means becoming a part of a global and diverse community dedicated to supporting our customers, communities, and each other. At American Express, every contribution is valued, and every colleague is recognized. We strive to provide the best customer...

  • Big Data Engineer

    1 week ago


    Gurgaon, Haryana, India Virtusa Full time

    Big Data Engineer - CREQ190572 Description Qualifications Bachelors degree in Engineering or Computer Science or equivalent OR Masters in Computer Applications or equivalent. 6+ years of software development experience and leading teams of engineers and scrum teams 3+ years of hands-on experience of working with Map-Reduce, Hive, Spark (core, SQL and...


  • Gurgaon, Haryana, India dunnhumby Full time

    We're looking for a Sr Big Data Engineer who expects more from their career. It's chance to extend and improve dunnhumby's Data Engineering Team. It's an opportunity to work with a market-leading business to explore new opportunities for us and influence global retailers. What we expect from you • 5+ years of experience in Information Technology...

  • Big Data Developer

    1 week ago


    Gurgaon, Haryana, India Impetus Full time

    Location: Gurgaon,Experience: 4- 6 yearsDescription:We are looking for Big Data developers having minimum experience of 4 year in Hadoop Ecosystem, passionate about technology, motivated for continuous learning and an individual who views every client interaction as an opportunity to create an exceptional customer experience.Qualifications:Must have:...