Data Engineer
3 weeks ago
Job Description : Data Engineer (Python). About Us : VCCEdge, part of Mosaic Digital is an upcoming company dedicated to building innovative products for private market stakeholders. Our platform provides valuable insights, analytics, and solutions to empower decision-makers to navigate the dynamic landscape of the private market. Overview: We are seeking an experienced Data Engineer proficient in Python to lead and optimize data engineering solutions. This role involves architecting, designing, and implementing scalable, high-performance data pipelines and web crawling solutions to extract, process, and analyze complex datasets. The ideal candidate will bring more than 4 years of experience, demonstrating expertise in managing large-scale data ecosystems, automation, and cloud-based data platforms. Responsibilities : - Architect and develop scalable data engineering solutions to manage, process, and transform large and diverse datasets efficiently. - Design and implement robust data pipelines using Python to ensure high availability, integrity, and quality of data. - Lead the development of web crawlers using Scrapy, BeautifulSoup, and other open-source frameworks for extracting data from various online sources. - Develop and optimize ETL/ELT pipelines, ensuring efficiency and reliability. - Integrate and manage data storage solutions, including MongoDB, MySQL, Elasticsearch, and Redis. - Implement and enhance automation scripts to streamline data collection, transformation, and normalization. - Monitor and optimize data processing workflows, identifying and resolving performance bottlenecks. - Ensure security, scalability, and compliance in data architecture and workflows. - Collaborate with cross-functional teams to integrate data insights into larger systems and products. - Develop and maintain APIs to facilitate data access, retrieval, and integration. - Lead data governance initiatives to maintain data consistency, quality, and accessibility. - Stay updated with the latest advancements in big data technologies, cloud computing (AWS/Azure), and data engineering best practices. Requirements : - Bachelor's or Master's degree in Computer Science, Engineering, or a related field. - 4 years of experience as a Data Engineer with a strong focus on Python. - Expertise in data engineering, ETL pipelines, data warehousing, and distributed systems. - Proven experience in designing high-performance databases and optimized data storage solutions. - Strong proficiency in web crawling and data extraction using Python libraries like Scrapy, BeautifulSoup, Selenium. - Experience with cloud platforms (AWS, Azure, or GCP) for data storage, processing, and analytics. - Solid understanding of SQL, NoSQL databases, and indexing strategies. - Experience with data processing frameworks such as Apache Spark, Kafka, or Airflow is a plus. - Strong problem-solving abilities and a detail-oriented approach to data architecture. - Ability to work independently and collaboratively in a fast-paced environment. (ref:hirist.tech)
-
Sr. Data Engineer
13 hours ago
India Data Vizionaries Full timeSenior Data EngineerLocation: Remote from India.Job Type: Part-time, India evening/night hours.Hours: U.S. Eastern Hours.If you don't meet even one of the above 6 requirements, please do not apply.1. Commit to 4 or 5 hours between 7/8pm and 11/12 IST (9:00–11:00 AM and 1:00–3:00 PM EST)2. Expertise in Azure, specifically ADF, Blob Storage, ADLS, Azure...
-
Sr. Data Engineer
17 hours ago
India Data Vizionaries Full timeSenior Data Engineer Location: Remote from India.Job Type: Part-time, India evening/night hours.Hours: U.S. Eastern Hours.If you don't meet even one of the above 6 requirements, please do not apply.Commit to 4 or 5 hours between 7/8pm and 11/12 IST (9:00–11:00 AM and 1:00–3:00 PM EST)Expertise in Azure, specifically ADF, Blob Storage, ADLS, Azure SQL and...
-
Data Engineer with a Focus on Scalability
12 hours ago
India Data Vizionaries Full timeAbout UsData Vizionaries is a leading data engineering and analytics company, dedicated to delivering innovative solutions that drive business growth.Job OverviewWe are seeking an experienced Senior Data Engineer to join our team on a part-time basis. As a key member of our team, you will be responsible for building and optimizing scalable data pipelines and...
-
India Data Vizionaries Full timeAbout UsData Vizionaries specializes in delivering end-to-end data engineering and analytics solutions that meet the evolving needs of businesses.Job DescriptionWe are looking for a skilled Senior Data Engineer to join our team on a part-time basis. As a key member of our team, you will be responsible for designing and developing scalable data pipelines and...
-
Data Engineer
3 days ago
India Mars Data Insights Full timeMars Data Insights is seeking a highly skilled Snowflake Developer to join our team. This role involves developing and maintaining complex data models using Snowflake and DBT, ensuring data quality and consistency across systems.The ideal candidate will have strong expertise in:Developing and maintaining DBT modelsUnderstanding and documenting data flow...
-
Senior Data Engineer
4 days ago
India Data Unveil Full timeAbout Data UnveilData Unveil is a forward-thinking organization dedicated to harnessing the power of data.We are seeking a skilled Python developer with 2-3 years of experience. The ideal candidate will have expertise in Python, PySpark, Oracle, MySQL, and basic AWS services.Main Responsibilities:Develop and optimize ETL workflows and data pipelines using...
-
Tatras Data
3 weeks ago
India Tatras Data Services Pvt Ltd Full timeJob Title : Data Engineer - Data Fabric/Bricks Location : Remote Job Summary : We are seeking an experienced Data Engineer to join our team as contractor. The ideal candidate will have a strong background in designing, implementing, and managing large-scale data platforms using Data Fabric or Data Bricks. The candidate should have experience working with...
-
Data Engineer
3 days ago
India Mars Data Insights Full timeMars Data Insights is a leading data-driven organization, seeking an experienced Snowflake Developer to join our team. The ideal candidate will possess strong expertise in DBT and SQL, with the ability to develop and maintain complex data models.The successful candidate will be responsible for ensuring data quality and consistency, performing rigorous data...
-
Cloud Data Modeling Professional
2 days ago
India Mars Data Insights Full timeMars Data Insights is expanding its team of Snowflake Developers and seeks an experienced professional to join our data engineering team.The successful candidate will be responsible for:Developing and maintaining Snowflake databasesDesigning and implementing data architecturesEnsuring data security and complianceCollaborating with cross-functional teams to...
-
Data Engineer
3 weeks ago
India Argus Consulting Full timeCompany Description. Argus Consulting (India) is a software consulting service firm with a focus on hi-tech technologies, including cloud-based technologies. The company is committed to delivering innovative solutions to clients in various industries. Role Description : - This is a remote contract role for a Senior Snowflake Data Engineer at Argus Consulting...
-
Data Engineer
3 weeks ago
India Anlage Infotech Pvt Ltd Full timeKey Responsibilities : - Framework Development & Optimization : Design and develop modular, reusable, and scalable Python-based frameworks to support data engineering and processing tasks. - Data Engineering & Transformation : Implement and maintain frameworks that facilitate the ingestion, transformation, and processing of large datasets using Azure...
-
Data Engineer
2 weeks ago
India Growel Softech Pvt. Ltd. Full timeRole : Data Engineer Description : We are seeking a seasoned Data Engineer to join our team in India. The ideal candidate will have extensive experience in building and optimizing data pipelines, ensuring data integrity, and collaborating with cross-functional teams to meet data needs. Responsibilities : - Design and implement data pipelines to support data...
-
Data Engineer
3 weeks ago
India MWIDM Staffing Services Full timeProfile : GCP Data Engineer. Location : Pan India (Hybrid Role). Experience : 6-9 Years. Notice Period : Immediate/ 30 Days Max. Job Description : The GCP Data Engineer plays a crucial role in designing, implementing, and managing data processing systems leveraging Google Cloud Platform (GCP) services. This position is essential for ensuring efficient data...
-
Data Engineer
2 weeks ago
India NexionPro Services Full timeJob Title: Data Engineer/Lead Data Engineer Experience Requirements: Data Engineer: 6-8 Years Lead Data Engineer: 9-13 Years Locations: Bengaluru, Pune, Hyderabad, Chennai, Jaipur, Gurugram Work Mode: Hybrid Job Description: We seek a skilled Data Engineer with expertise in Spark, Snowflake, Hive , and other big data technologies to join our dynamic...
-
Data Engineer
3 weeks ago
India Risk Resources LLP Full timeOverview : The Data Engineer is a crucial member of our data team, responsible for designing, building, and maintaining robust data infrastructure. Utilizing platforms such as Databricks and Azure Data Factory, this role focuses on creating efficient data pipelines and managing large datasets for analysis. By employing tools like PySpark and SQL, the Data...
-
Data Engineer
2 weeks ago
India Vintronics Consulting Full timeWe are looking for a highly motivated and skilled Data Engineer to join our growing data team. As a Data Engineer, you will play a crucial role in designing, building, and maintaining scalable data pipelines that drive our data ecosystem. You will work closely with data scientists, analysts, and software engineers to deliver high-quality data solutions that...
-
Data Engineer
2 weeks ago
India Auxo AI Full timeRole Summary: We are seeking skilled Data Engineers with prior experience on Azure data engineering services and SQL server to join our team. As a Data Engineer, you will be responsible for designing and implementing robust data infrastructure, building scalable data pipelines, and ensuring efficient data integration and Design, develop, and maintain...
-
Data Engineer
3 weeks ago
India Algoscale Technologies Full timeJob Summary : We are seeking a skilled and motivated Data Engineer to join our team. The ideal candidate will design, develop, and maintain scalable data pipelines, optimize data storage solutions, and ensure efficient data integration across multiple systems. You will play a critical role in building the data infrastructure that supports our business...
-
Data engineer
2 weeks ago
India OMEGA HIRES CONSULTANCY Full timeJob Title : GCP Data engineer Duration : 03 Months Contract to Hire Position, Deployed at client location Job Summary : We are looking for a skilled GCP Data Engineer to join our dynamic team. In this role, you will be responsible for building, managing, and optimizing data pipelines in Google Cloud Platform (GCP) to enable data-driven decisions and...
-
Data Engineer
7 days ago
India MWIDM Staffing Services Full timeResponsibilities :- Design, develop, and maintain efficient and reliable data pipelines using PySpark.- Implement ETL processes to ingest, transform, and load data from various sources.- Optimize PySpark code for performance and scalability.- Work with large datasets and distributed computing frameworks.- Develop and maintain data quality checks and...