Data Engineer

4 weeks ago

Gurgaon, Haryana, India HT Digital Streams Full time

Job Description : Data Engineer (Python).

About Us :

VCCEdge, part of Mosaic Digital is an upcoming company dedicated to building innovative products for private market stakeholders.

Our platform provides valuable insights, analytics, and solutions to empower decision-makers to navigate the dynamic landscape of the private market.

Overview: We are seeking an experienced Data Engineer proficient in Python to lead and optimize data engineering solutions.

This role involves architecting, designing, and implementing scalable, high-performance data pipelines and web crawling solutions to extract, process, and analyze complex datasets.

The ideal candidate will bring more than 4 years of experience, demonstrating expertise in managing large-scale data ecosystems, automation, and cloud-based data platforms.

Responsibilities :

- Architect and develop scalable data engineering solutions to manage, process, and transform large and diverse datasets efficiently.

- Design and implement robust data pipelines using Python to ensure high availability, integrity, and quality of data.

- Lead the development of web crawlers using Scrapy, BeautifulSoup, and other open-source frameworks for extracting data from various online sources.

- Develop and optimize ETL/ELT pipelines, ensuring efficiency and reliability.

- Integrate and manage data storage solutions, including MongoDB, MySQL, Elasticsearch, and Redis.

- Implement and enhance automation scripts to streamline data collection, transformation, and normalization.

- Monitor and optimize data processing workflows, identifying and resolving performance bottlenecks.

- Ensure security, scalability, and compliance in data architecture and workflows.

- Collaborate with cross-functional teams to integrate data insights into larger systems and products.

- Develop and maintain APIs to facilitate data access, retrieval, and integration.

- Lead data governance initiatives to maintain data consistency, quality, and accessibility.

- Stay updated with the latest advancements in big data technologies, cloud computing (AWS/Azure), and data engineering best practices.

Requirements :

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

- 4+ years of experience as a Data Engineer with a strong focus on Python.

- Expertise in data engineering, ETL pipelines, data warehousing, and distributed systems.

- Proven experience in designing high-performance databases and optimized data storage solutions.

- Strong proficiency in web crawling and data extraction using Python libraries like Scrapy, BeautifulSoup, Selenium.

- Experience with cloud platforms (AWS, Azure, or GCP) for data storage, processing, and analytics.

- Solid understanding of SQL, NoSQL databases, and indexing strategies.

- Experience with data processing frameworks such as Apache Spark, Kafka, or Airflow is a plus.

- Strong problem-solving abilities and a detail-oriented approach to data architecture.

- Ability to work independently and collaboratively in a fast-paced environment.

(ref:hirist.tech)

  • Analyst

    4 days ago

    Gurgaon, Haryana, India Data Engineer Full time

    Job Title: Data Engineer (ML & GenAI Deployment)We are seeking a highly skilled Data Engineer to join our team. As a key member, you will be responsible for deploying Machine Learning, Vision Language Models, and Large Language Models into production using cloud platforms.Main Responsibilities:Data Pipelines Management: Build and manage data pipelines to...

  • Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India Eucloid Data Solutions Full time

    Job descriptionWe are looking for a high-energy individual for the Data Engineer role at Eucloid. The candidate will advise clients on multiple business problems and help them achieve desirable business outcomes through projects. The candidate is expected to be a highly motivated individual with an ability to provide strategic operational leadership for a...

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India Eucloid Data Solutions Full time

    Job description We are looking for a high-energy individual for the Data Engineer role at Eucloid. The candidate will advise clients on multiple business problems and help them achieve desirable business outcomes through projects. The candidate is expected to be a highly motivated individual with an ability to provide strategic operational leadership for a...

  • Application Performance Engineer

    4 days ago

    Gurgaon, Haryana, India NTT DATA Full time

    Responsibilities">As a Production Support Engineer at NTT DATA, you will be responsible for ensuring the smooth operation of our production systems. This includes:">">Maintaining and troubleshooting production systems">Monitoring and analyzing system performance">Identifying and resolving issues in a timely manner">Collaborating with development teams to...

  • Lead Data Analyst

    3 weeks ago

    Gurgaon, Haryana, India Eucloid Data Solutions Full time

    Job description:We are looking for a high-energy individual for the role of Lead Data Consultant at Eucloid. The candidate will collaborate with client stakeholders on multiple projects in Marketing Analytics vertical.The candidate is expected to lead following work-streams:• Identifying opportunities and influencing partners to act: You will partner with...

  • Network engineer

    13 hours ago

    Gurgaon, Haryana, India Real Time Data Services Full time

    Job Title:Network EngineerSummary:We are seeking an experienced Network Engineer to join our team at Real Time Data Services. The successful candidate will have a strong background in network engineering, with expertise in cloud security, firewall management, and networking infrastructure.Responsibilities:Configuring and monitoring firewalls, switches, and...

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India beBee Careers Full time

    Data Engineer PositionWe are seeking a skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for developing, optimizing, and maintaining scalable and reliable data pipelines using tools such as Python, SQL, and Spark.

  • Data Engineer

    7 days ago

    Gurgaon, Haryana, India beBee Careers Full time

    Job DescriptionThe Data Engineer plays a pivotal role in designing, implementing, and optimizing data ingestion, transformation pipelines and processes.Key Responsibilities:Create scalable, efficient, and reliable data pipelines using cloud technologies like AWS, Azure, or GCP.Develop and implement data models, processing pipelines, and back-end services to...

  • Data Engineer

    18 hours ago

    Gurgaon, Haryana, India TALENT XO Full time

    Roles and Responsibilities : 1. Design, develop, and implement complex data pipelines and transformations using Azure, SQL, Spark, and Python to support data processing and analytics projects.2. Collaborate with cross-functional teams to gather requirements for data solutions and provide technical expertise in designing and implementing Presales...

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India beBee Careers Full time

    **Data Engineer Position**About the RoleWe are seeking an experienced Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems.Key ResponsibilitiesDesign and operationalize large-scale data solutions using cloud services like Data Flow, Data Proc, Pub Sub, BigQuery, Cloud...