High-Performance Data Solutions Architect
6 days ago
We're seeking a highly motivated and driven professional to own the entire tech stack, shape technology vision and architecture, and build innovative scalable solutions from day zero.
This role is ideal for someone who thrives on early-stage challenges, loves building high-performance solutions, and is eager to make a significant impact in data engineering.
Key Responsibilities:- Web Scraping & Crawling: Develop automated scrapers to extract structured and unstructured data from websites, APIs, and public datasets.
- Scalable Scraping Systems: Build multi-threaded, distributed crawlers capable of handling high-volume data collection without interruptions.
- Data Parsing & Cleaning: Normalize scraped data, remove noise, and ensure consistency before passing to data pipelines.
- Anti-bot & Evasion Tactics: Implement proxy rotation, captcha solving, and request throttling techniques to handle scraping restrictions.
- Integration with Pipelines: Deliver clean, structured datasets into NoSQL stores and ETL pipelines for further enrichment and graph-based storage.
- Data Quality & Validation: Ensure data accuracy, deduplicate records, and maintain a trust scoring system for data confidence.
- Documentation & Maintenance: Keep scrapers updated when websites change, and document scraping logic for reproducibility.
- 2+ years of experience in web scraping, crawling, or data collection.
- Strong proficiency in Python (libraries like BeautifulSoup, Scrapy, Selenium, Playwright, Requests).
- Familiarity with NoSQL databases (MongoDB, DynamoDB) and data serialization formats (JSON, CSV, Parquet).
- Experience in handling large-scale scraping with proxy management and rate-limiting.
- Basic knowledge of ETL processes and integration with data pipelines.
- Exposure to graph databases (Neo4j) is a plus.
- Data Quality & Validation: Ensure data accuracy, deduplicate records, and maintain a trust scoring system for data confidence.
- Integration with Pipelines: Deliver clean, structured datasets into NoSQL stores and ETL pipelines for further enrichment and graph-based storage.
- Anti-bot & Evasion Tactics: Implement proxy rotation, captcha solving, and request throttling techniques to handle scraping restrictions.
- Data Parsing & Cleaning: Normalize scraped data, remove noise, and ensure consistency before passing to data pipelines.
- Scalable Scraping Systems: Build multi-threaded, distributed crawlers capable of handling high-volume data collection without interruptions.
We value transparency, collaboration, and individual ownership, creating an environment where talented people can do their best work.
We prioritize problem-driven innovation, rapid action, and collaborative excellence to drive our success.
- Problem-Driven Innovation: We're deeply committed to solving real challenges that genuinely matter.
- Rapid Action & Ownership: We encourage autonomy and accountability—own your projects, move quickly, and shape the future.
- Collaborative Excellence: Cross-team collaboration ensures alignment, sparks innovation, and drives us forward together.
-
High-Performance Data Migration Specialist
2 weeks ago
Tiruppur, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,90,000 - ₹ 21,94,000Azure Data Engineer RoleWe are seeking a skilled Azure data engineer with hands-on experience in modern data engineering tools and platforms within the Azure ecosystem. Our ideal candidate will have a strong foundation in data integration, transformation, and migration.This individual will lead large-scale SQL workload migrations from on-premise environments...
-
Tiruppur, Tamil Nadu, India beBeeDatabase Full time ₹ 12,00,000 - ₹ 16,00,000Database EngineerWe are seeking a highly skilled and motivated Database Engineer to join our team, focusing on the core infrastructure that powers our financial trading and analysis platforms.Design, implement, and maintain database schemas for storing and retrieving time-series financial data.Develop and optimize ETL processes for ingesting data from...
-
High-Performance Data Engineer
6 days ago
Tiruppur, Tamil Nadu, India beBeeData Full time US$ 1,30,000 - US$ 1,70,000Job Title: Data ArchitectAbout the Role:We are seeking a highly skilled Data Architect to design and build high-performance data pipelines that process large volumes of data.Key Responsibilities:Design scalable batch processing systems using Python and big data technologies.Optimize database performance focusing on slow-running queries and latency...
-
High-Performance Systems Architect
1 week ago
Tiruppur, Tamil Nadu, India beBeeLogicEngineer Full time ₹ 1,80,00,000 - ₹ 2,20,00,000Our team is seeking a high-caliber Senior Logic Design Engineer to join our Core Units Front End Pipeline group.Job DescriptionThis role will involve designing and architecting high-performance systems, driving innovation in the quantum revolution and AI era. We are looking for an expert who can develop features, estimate effort, silicon area, and wire...
-
Data Solutions Architect
1 week ago
Tiruppur, Tamil Nadu, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000About the RoleWe are seeking a highly skilled Data Architect to design and implement scalable data solutions on AWS.Location: BangaloreWork Model: HybridExperience: 5-7 yearsNotice Period: Immediate- 15 daysOpen roles: 24Data Architecture, Java, SQL, Data Modeling, ETL/ELTJob Description:Design and implement scalable data solutions on AWS (S3, Redshift,...
-
Databricks Solutions Architect
2 weeks ago
Tiruppur, Tamil Nadu, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,00,00,000Job Title: Databricks Solutions ArchitectWe are seeking a seasoned professional to design and develop scalable, high-performance data solutions using Databricks.The ideal candidate will have a deep understanding of modern data architecture, hands-on experience in Databricks production environments, and strong expertise in data engineering principles.The...
-
Senior Data Architect
2 weeks ago
Tiruppur, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000Job Overview:We are seeking a highly skilled and experienced Senior Data Architect to join our team. This is a key role in which you will be responsible for designing, architecting, and implementing distributed data solutions for our clients.In this position, you will work closely with our customers to understand their business needs and develop tailored...
-
Enterprise Data Solutions Architect
1 week ago
Tiruppur, Tamil Nadu, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000Job Title: Enterprise Data Solutions ArchitectWe are seeking a highly skilled Enterprise Data Solutions Architect with 6–8 years of experience in designing, developing, and managing enterprise data solutions on Google Cloud Platform (GCP). The ideal candidate will have a strong background in cloud data architecture, data warehousing, big data processing,...
-
Principal Enterprise Solutions Architect
1 week ago
Tiruppur, Tamil Nadu, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000Enterprise Data ArchitectWe are seeking a seasoned data architect to design and implement large-scale data solutions.Develop an overall data and solution architecture for large-scale enterprise projects.Create scalable, secure, and high-performing data architectures aligned with business goals.Lead evaluation and integration of data platforms, tools, and...
-
Chief Data Architect
1 week ago
Tiruppur, Tamil Nadu, India beBeeDataEngineer Full time ₹ 1,44,00,000 - ₹ 2,24,80,000Job Overview:We are seeking a seasoned data professional to lead the design and implementation of scalable, high-performance data pipelines using Snowflake and dbt. The ideal candidate will possess expertise in architecting modular ELT pipelines, designing layered data models, and optimizing Snowflake for performance and cost.This role requires strong...
