Data Engineer

4 weeks ago

Gurugram, India HT Digital Streams Full time

Job Description : Data Engineer (Python).

About Us :


VCCEdge, part of Mosaic Digital is an upcoming company dedicated to building innovative products for private market stakeholders.

Our platform provides valuable insights, analytics, and solutions to empower decision-makers to navigate the dynamic landscape of the private market.

Overview: We are seeking an experienced Data Engineer proficient in Python to lead and optimize data engineering solutions.

This role involves architecting, designing, and implementing scalable, high-performance data pipelines and web crawling solutions to extract, process, and analyze complex datasets.

The ideal candidate will bring more than 4 years of experience, demonstrating expertise in managing large-scale data ecosystems, automation, and cloud-based data platforms.

Responsibilities :

- Architect and develop scalable data engineering solutions to manage, process, and transform large and diverse datasets efficiently.

- Design and implement robust data pipelines using Python to ensure high availability, integrity, and quality of data.

- Lead the development of web crawlers using Scrapy, BeautifulSoup, and other open-source frameworks for extracting data from various online sources.

- Develop and optimize ETL/ELT pipelines, ensuring efficiency and reliability.

- Integrate and manage data storage solutions, including MongoDB, MySQL, Elasticsearch, and Redis.

- Implement and enhance automation scripts to streamline data collection, transformation, and normalization.

- Monitor and optimize data processing workflows, identifying and resolving performance bottlenecks.

- Ensure security, scalability, and compliance in data architecture and workflows.

- Collaborate with cross-functional teams to integrate data insights into larger systems and products.

- Develop and maintain APIs to facilitate data access, retrieval, and integration.

- Lead data governance initiatives to maintain data consistency, quality, and accessibility.

- Stay updated with the latest advancements in big data technologies, cloud computing (AWS/Azure), and data engineering best practices.

Requirements :

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

- 4+ years of experience as a Data Engineer with a strong focus on Python.

- Expertise in data engineering, ETL pipelines, data warehousing, and distributed systems.

- Proven experience in designing high-performance databases and optimized data storage solutions.

- Strong proficiency in web crawling and data extraction using Python libraries like Scrapy, BeautifulSoup, Selenium.

- Experience with cloud platforms (AWS, Azure, or GCP) for data storage, processing, and analytics.

- Solid understanding of SQL, NoSQL databases, and indexing strategies.

- Experience with data processing frameworks such as Apache Spark, Kafka, or Airflow is a plus.

- Strong problem-solving abilities and a detail-oriented approach to data architecture.

- Ability to work independently and collaboratively in a fast-paced environment.

(ref:hirist.tech)

  • Junior Data Engineer

    4 weeks ago

    Gurugram, India Pacific Data Integrators Full time

    Junior Data Engineer (Immediate Joiner Only)Location: RemoteWorking Hours: 5 PM – 2 AM IST shiftMUST be ready to join in 15 days notice MaxMUST have good comm skillsMUST have experience between 1-2 years MAX, not more than thatPacific Data Integrators (PDI) is currently hiring JUNIOR ETL / DATA ENGINEER (GURUGRAM) for its enterprise client projects...

  • Junior Data Engineer

    4 weeks ago

    Gurugram, India Pacific Data Integrators Full time

    Junior Data Engineer (Immediate Joiner Only) Location: Remote Working Hours: 5 PM – 2 AM IST shift MUST be ready to join in 15 days notice Max MUST have good comm skills MUST have experience between 1-2 years MAX, not more than that Pacific Data Integrators (PDI) is currently hiring JUNIOR ETL / DATA ENGINEER (GURUGRAM) for its...

  • Junior Data Engineer

    4 weeks ago

    Gurugram, India Pacific Data Integrators Full time

    Junior Data Engineer (Immediate Joiner Only) Location: Remote Working Hours: 5 PM – 2 AM IST shift MUST be ready to join in 15 days notice Max MUST have good comm skills MUST have experience between 1-2 years MAX, not more than that Pacific Data Integrators (PDI) is currently hiring JUNIOR ETL / DATA ENGINEER (GURUGRAM) for its...

  • Data Engineer

    4 weeks ago

    Gurugram, India Cognitio Analytics Inc Full time

    Cognitio Analytics, founded in 2013, aims to be the preferred provider of AI / ML driven productivity solutions for large enterprises. The company has received awards for its Smart Operations and Total Rewards Analytics Solutions and is dedicated to innovation, R&D, and creating sustained value for clients. Cognitio Analytics has been recognized as a "Great...

  • Data Engineer

    4 weeks ago

    Gurugram, India PepsiCo Full time

    Responsibilities As a member of the data engineering team, you will be the key technical expert developing and overseeing PepsiCo's data product build & operations and drive a strong vision for how data engineering can proactively create a positive impact on the business. You'll be an empowered member of a team of data engineers who build data pipelines into...

  • Data Engineer

    4 weeks ago

    Gurugram, India PepsiCo Full time

    ResponsibilitiesAs a member of the data engineering team, you will be the key technical expert developing and overseeing PepsiCo's data product build & operations and drive a strong vision for how data engineering can proactively create a positive impact on the business. You'll be an empowered member of a team of data engineers who build data pipelines into...

  • Data Engineer

    4 weeks ago

    Gurugram, India PepsiCo Full time

    Responsibilities As a member of the data engineering team, you will be the key technical expert developing and overseeing PepsiCo's data product build & operations and drive a strong vision for how data engineering can proactively create a positive impact on the business. You'll be an empowered member of a team of data engineers who build data pipelines into...

  • Data Engineer

    4 weeks ago

    gurugram, India Evalueserve India Full time

    About Data Analytics (DA) Data Analytics is one of the highest growth practices within Evalueserve, providing you rewarding career opportunities. Established in 2014, the global DA team extends beyond 1000+ (and growing) data science professionals across data engineering, business intelligence, digital marketing, advanced analytics, technology, and product...

  • Data Engineer

    4 weeks ago

    gurugram, India RSA India (An Intact Company) Full time

    Data Engineer About Us: RSA India (an Intact Company) is a financial services company. We are strong, and well-established and have been protecting individuals and businesses from risk for over 300 years. We are here to help people, business and society prosper in good times and be resilient in bad times in bad times. We created a purpose-driven company...

  • Data Engineer

    4 weeks ago

    Gurugram, India EliteRecruitments Full time

    Job Title: Data Engineer with Scala & Spark Locations: Pune/Gurgaon/Bangalore Job Description: As a Data Engineer with Scala & Spark, you will be responsible for designing, developing, and maintaining Scala applications. You will collaborate with cross-functional teams to define, design, and ship new features, as well as maintain and improve existing...

  • Data Engineer

    4 weeks ago

    Gurugram, India EXL Full time

    About the Role: We are looking for a skilled Python Data Engineer to design, build, and maintain scalable data pipelines and infrastructure. You will collaborate with cross-functional teams to ensure efficient data flow and accessibility for analytics and decision-making. Key Responsibilities: Develop and optimize ETL processes and data pipelines using...

  • Lead Data Engineer

    5 days ago

    Gurugram, India Antal International Full time

    Job Description Lead Data engineer helps to architect, design, implement, and optimise our data ingestion, transformation pipelines and processes. He or she develops data models, processing pipelines, and back-end services supporting the data science teams, automates processes, builds integrations, and builds analytics. Desired skills: · A...

  • Data Engineer

    4 weeks ago

    Gurugram, India BayOne Solutions Full time

    Min 5 years of experience in modern data engineering/data warehousing/data lakes technologies on cloud platforms like Azure, Data Bricks etc.5 + years of proven experience with SQL, schema design and dimensional data modellingSolid knowledge of data warehouse best practices, development standards and methodologiesExperience with ETL/ELT tools like ADF,...

  • Data Engineer

    4 weeks ago

    Gurugram, India BayOne Solutions Full time

    Min 5 years of experience in modern data engineering/data warehousing/data lakes technologies on cloud platforms like Azure, Data Bricks etc. 5 + years of proven experience with SQL, schema design and dimensional data modelling Solid knowledge of data warehouse best practices, development standards and methodologies Experience with ETL/ELT tools like ADF,...

  • Cloud Data Engineer

    4 weeks ago

    gurugram, India RiDiK (a Subsidiary of CLPS. Nasdaq: CLPS) Full time

    Job Title: Cloud Data Engineer – Sr. Software Engineer. Location: Gurugram, India Experience: 6 - 13 years We are looking for Immediate joiner candidates (Max 30days). Job Summary: We are seeking an experienced Cloud Data Engineer – Sr. Software Engineer with 6 to 13 years of expertise in Python for Data Analysis . The ideal candidate should be...

  • Data Engineer

    4 weeks ago

    Gurugram, India BayOne Solutions Full time

    Work from office - 3 days/WeekLocation: Bangalore/GurgaonJob Description:Min 5 years of experience in modern data engineering/data warehousing/data lakes technologies on cloud platforms like Azure, AWS, GCP, Data Bricks etc.Azure experience is preferred over other cloud platforms.5 years of proven experience with SQL, schema design and dimensional data...

  • Data Engineer

    4 weeks ago

    Gurugram, India BayOne Solutions Full time

    Work from office - 3 days/WeekLocation: Bangalore/GurgaonJob Description:Min 5 years of experience in modern data engineering/data warehousing/data lakes technologies on cloud platforms like Azure, AWS, GCP, Data Bricks etc.Azure experience is preferred over other cloud platforms.5 years of proven experience with SQL, schema design and dimensional data...

  • Data Engineer

    4 weeks ago

    Gurugram, India Gravity Infosolutions Full time

    Job Title: Senior Data Engineer (Contexa, AWS, Spark) Location : Gurugram, India (Hybrid – 3 days in office, 2 days WFH) ⏳ Term : 12+ months Job Overview: We are seeking a Senior Data Engineer with expertise in VMware Contexa, Spark, AWS, and ETL for a high-impact role in the banking & finance sector . The ideal candidate will have strong...

  • Data Engineer

    4 weeks ago

    Gurugram, India Gravity Infosolutions Full time

    Job Title: Senior Data Engineer (Contexa, AWS, Spark)📍 Location: Gurugram, India (Hybrid – 3 days in office, 2 days WFH)⏳ Term: 12+ monthsJob Overview:We are seeking a Senior Data Engineer with expertise in VMware Contexa, Spark, AWS, and ETL for a high-impact role in the banking & finance sector. The ideal candidate will have strong experience in big...

  • Data Engineer

    4 weeks ago

    Gurugram, India EliteRecruitments Full time

    Job Title: Data Engineer with Scala & SparkLocations: Pune/Gurgaon/BangaloreJob Description:As a Data Engineer with Scala & Spark, you will be responsible for designing, developing, and maintaining Scala applications. You will collaborate with cross-functional teams to define, design, and ship new features, as well as maintain and improve existing codebases....