Data Engineer/Manager

1 week ago

Gurgaon, Haryana, India HuQuo Consulting Pvt. Ltd. Full time

GCP Data Engineer role involves designing, developing, and maintaining data pipelines and infrastructure on Google Cloud Platform (GCP). The position demands expertise in data engineering practices, cloud architecture, and big data technologies. The ideal candidate collaborates with data scientists, analysts, and stakeholders to ensure efficient, reliable data systems for informed decision-making within the organization.



Key Responsibilities:

Data Pipeline Development:

  • Create and manage scalable ETL/ELT pipelines on GCP.
  • Establish data ingestion processes from diverse sources (e.g., APIs, databases, file systems).
  • Maintain data quality, integrity, and reliability throughout its lifecycle.

Cloud Architecture:

  • Design GCP data architecture using services like BigQuery, Dataflow, Pub/Sub, Cloud Storage, and Cloud Composer.
  • Optimize data storage and retrieval for performance and cost efficiency.
  • Ensure secure, scalable data infrastructure aligned with industry standards.

Big Data Processing:

  • Develop and oversee largescale data processing workflows with Apache Beam, Dataflow, and other technologies.
  • Implement real-time data streaming via Pub/Sub and Dataflow.
  • Enhance data processing jobs for improved performance and cost-effectiveness.

Collaboration and Communication:

  • Engage with data scientists, analysts, and stakeholders to grasp data requirements and offer tailored solutions.
  • Effectively communicate technical concepts to diverse audiences.
  • Participate in agile development processes including sprint planning and retrospectives.

Data Management and Governance:

  • Establish and uphold data governance practices including cataloging, metadata management, and lineage.
  • Ensure compliance with data security and privacy regulations.
  • Monitor and maintain data quality and coherence.

Troubleshooting and Support:

  • Identify and resolve technical issues in data pipelines and infrastructure.
  • Provide support and upkeep for existing data solutions.
  • Continuously enhance data pipeline performance and efficiency.

Experience:

  • Minimum 5-7 years of experience in data engineering.
  • Demonstrated expertise with GCP data services and tools.

Technical Skills:

  • Proficient in GCP services (e.g., BigQuery, Dataflow, Pub/Sub, Cloud Storage, Cloud Composer).
  • Strong programming skills especially in Python.
  • Familiar with big data technologies and frameworks (e.g., Apache Beam, Hadoop, Spark).
  • Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes) is advantageous.

Key Competencies:

  • Strong problem-solving and attention to detail.
  • Excellent communication and teamwork abilities.
  • Adaptability to fast-paced, dynamic environments.
  • Self-motivated and capable of independent and team work.
  • Continuously learning mindset with a passion for emerging technologies.

  • Data Engineering Manager

    1 week ago

    Gurgaon, Haryana, India Skyleaf Full time

    We are looking for an experienced Azure Cloud and Data Engineering Manager to support our Enterprise Data Environment and provide strategic and technical direction to help deliver solutions using cloud-based : Manager - Azure Cloud and Data Engineering Manager Experience : 10-14 Years Location : Gurugram (Hybrid) Shift Timing : 2PM-11PM Key...

  • Manager/Data Engineer

    1 week ago

    Gurgaon, Haryana, India HuQuo Consulting Pvt. Ltd. Full time

    Job Description : The Data Engineering Team primarily work with the lead data engineer, product owners, architects, engineering teams and BI developers to implement and drive forward data & analytics goals within Risk. This is an opportunity to work as a "hands on" data engineer, engaging with a variety of stakeholders in engineering teams and the business...

  • Data Engineering Manager

    1 week ago

    Gurgaon, Haryana, India GEMINI Full time

    Department : Analytics Data is central to all of our business functions and drive many of our most important decisions at Gemini. As a result, the analysts, scientists, and engineers that make up the Data team are uniquely positioned to advise and influence cross-functional projects. These projects cover a wide-range of topics including product strategy,...

  • Data Engineering Manager

    1 week ago

    Gurgaon, Haryana, India HuQuo Full time

    Manager - Data EngineerJob Responsibilities :Create and maintain optimal data pipeline architecture, Assemble large, complex data sets that meet functional/nonfunctional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, redesigning infrastructure for greater...

  • Manager - Data Engineering

    1 week ago

    Gurgaon, Haryana, India Spectral Consultant Full time

    Spectral Consultants is looking for a Data Engineering Manager to join their team at a financial services organization.Experience: 10+ yearsRequirements:Extensive experience with Azure Cloud strategies & industry best practicesStrong skills in data management, data modeling, data services, reporting, and analytics toolsProficiency with SQL Server, Mongo DB,...

  • Data Engineering Manager

    1 week ago

    Gurgaon, Haryana, India Spectral Consultant Full time

    Spectral Consultants is looking for a Data Engineering- Manager to join a renowned financial services organization.Requirements:Experience: 10+ yearsExpertise in data management, data modeling, data services, reporting, and analytics.Familiarity with SQL Server, Mongo DB, PostgreSQL, and Azure Cloud applications.Proficiency in selfservice Business...

  • Manager - Data Engineering - Big Data/Azure

    1 week ago

    Gurgaon, Haryana, India Publicis Sapient Full time

    Job Description : As Manager, Data Engineering, you will be responsible for translating client requirements into design, architecting, and implementing Azure Cloud based big data solutions for clients. Your role will be focused on delivering high quality solutions by independently driving design discussions related to Data Ingestion, Transformation &...

  • Manager - Senior Data Engineer

    1 week ago

    Gurgaon, Haryana, India Macquarie Group Full time

    Senior Data Engineer Join our tooling division, focused on uplifting data management, regulatory reporting, and record management capabilities across Macquarie. At Macquarie, we are working to create lasting value for our communities, our clients, and our people. We are a global financial services group operating in 34 markets and with 54 years of...

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India HuQuo Consulting Pvt. Ltd. Full time

    Role Overview : We are looking for a go getter person who can define: Analytics strategies to meet the demands of business requirements Technical requirements of the analytic solutions Data requirements of the analytic solution processes The person will be part of the Data Science and Engineering team for a major Insurance client. He/ She will work with...

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India HuQuo Consulting Pvt. Ltd. Full time

    Job Description :Role Overview :We are looking for a go-getter person who can define :Data strategies to meet the demands of business requirements Technical requirements of the organizational data needs from diverse teams Data architecture, models and goforward data solutions for various teams of the organization The person will be part of the Data Solutions...

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India TransOrg Analytics Full time

    Location : Gurgaon/Mumbai Job ID : DE2008 Job Type : Full time Experience : 2-6 years Why would you like to join us? TransOrg Analytics, an award winning – Big Data and Predictive analytics- company, offers advanced analytics solutions to industry leaders and Fortune 500 companies across India, US, UK, Singapore and the Middle East. Our...

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India Teleparty Full time

    Teleparty is hiring Data Engineers Join Teleparty, a premium remote entertainment service leading the charge in making social TV a global cultural phenomenon bigger than the movies. With over 10 million streamers who have installed our Netflix focused browser extension - Teleparty, we operate the largest social TV service in the world for millions of...

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India HuQuo Consulting Pvt. Ltd. Full time

    Role & Responsibilities Overview : A seasoned data team member with a good grasp of both business and technical aspects.Good familiarity with data stack Interact with the other members of the data governance team, business teams to understand requirements and also collaborate with stewards / data custodians / end users etc Demonstrated experience of building...

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India Unifocus Full time

    Job Description Data Engineer Unifocus is an integrated workforce management software platform offering intelligent automation for daily work orders management, housekeeping activities, facility maintenance, survey solutions, scheduling & labour management, and time & attendance built for the hospitality market and other dynamic scheduling environment. We...

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India Ezeiatech Systems Private Limited Full time

    Roles & Responsibilities : Creating and managing data pipelines from various disparate sources. Leveraging Big Data technologies to enhance our home-grown Data Platform. Ensuring a high degree of data quality, scalability, and availability. Working on PoC's for selected technologies that are yet to be adopted. Working closely with Data Scientists to...

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India dentsu Full time

    The purpose of this role is to maintain, improve, clean and manipulate data in the business's operational and analytics databases. The Data Engineer works with the business's software engineers, data analytics teams, data scientists. data warehouse engineers and business intelligence experts and data visualization specialists to understand and aid in the...

  • Lead Data Engineer

    1 week ago

    Gurgaon, Haryana, India HuQuo Consulting Pvt. Ltd. Full time

    Job Description for this position include :Provides technical leadership in Big Data space (Hadoop Stack like M/R, HDFS, Pig, Hive, HBase, Flume, Sqoop, .NoSQL stores like Cassandra, HBase etc) across Fractal and contributes to open source Big Data technologies Write and tune complex Java, MapReduce, Pig and Hive jobs Adapt quickly to change in requirements...

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India Paisabazaar Full time

    Job Description : Qualifications for Data Engineer : 3+ Years of experience in building and optimizing 'big data' solutions required to fulfill business and technology requirements. 3+ years of technical expertise in areas of design and implementation using big data technology Hadoop, Hive, Spark, Python/Java. Strong analytic skills to understand and create...

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India Acuity Knowledge Partners Full time

    Opportunity- Data Engineer ( Associate/ Senior Associate) Location- Gurugram (Hybrid) "Immediate joiners are highly preferred " Key Responsibilities: Partner with data strategy and sourcing team on data requirements to design data pipelines and delivery structures. Engage with vendors and technical teams to systematically ingest, evaluate, and create...

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India HuQuo Full time

    As Data Engineer, your specific responsibilities include the following :Create and maintain optimal data pipeline architecture Assemble large, complex data sets that meet functional / nonfunctional business requirements Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, redesigning...