Building Scalable Solutions

22 hours ago

Ellore, Andhra Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

We are seeking a skilled Data Engineer to help build our technology platform from the ground up. As a key team member, you will shape our tech vision, architecture, and engineering culture while contributing to foundational development.

This role is ideal for someone eager to own the entire tech stack and thrive in an early-stage environment. You will be responsible for building innovative, scalable solutions and establishing best practices.

Key Responsibilities:

  • Web Scraping & Crawling
    • Design and maintain automated scrapers to extract data from websites, APIs, and public datasets.
      • Create multi-threaded, distributed crawlers for high-volume data collection without interruptions.
        • Parsing and Cleaning
          • Normalize scraped data, remove noise, and ensure consistency before passing it to data pipelines.
            • Evasion Tactics
              • Implement proxy rotation, captcha solving, and request throttling techniques to handle scraping restrictions.
                • Integration with Pipelines
                  • Deliver clean, structured datasets into NoSQL stores and ETL pipelines for further enrichment and graph-based storage.
                    • Data Quality & Validation
                      • Ensure data accuracy, deduplicate records, and maintain a trust scoring system for data confidence.
                        • Maintenance and Documentation
                          • Keep scrapers updated when websites change and document scraping logic for reproducibility.

Requirements:

You will need:

Technical Skills:

  • 2+ years of experience in web scraping, crawling, or data collection.
  • Strong proficiency in Python (BeautifulSoup, Scrapy, Selenium, Playwright, Requests).
  • Familiarity with NoSQL databases (MongoDB, DynamoDB) and data serialization formats (JSON, CSV, Parquet).
  • Experience in handling large-scale scraping with proxy management and rate-limiting.
  • Basic knowledge of ETL processes and integration with data pipelines.
  • Exposure to graph databases (Neo4j) is a plus.

Soft Skills:

  • Detail-oriented with a focus on accuracy and reliability of collected data.
  • Strong problem-solving skills, particularly in adapting scrapers to evolving web structures.
  • A curious mindset with a drive to discover new data sources.
  • Comfortable working in a fast-paced, early-stage startup environment.

About Us:

Our company values transparency, collaboration, and individual ownership, creating an environment where talented people can do their best work.

  • Build Scalable Data Systems

    7 days ago

    Ellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    We're seeking a skilled Data Engineer to join our team of innovators.The ideal candidate will have hands-on experience with designing and building efficient data pipelines, collaborating closely with cross-functional teams to ensure data is clean, reliable, and accessible for analytics, reporting, and product innovation.Key Responsibilities:Create scalable...

  • Building Scalable AI Systems

    20 hours ago

    Ellore, Andhra Pradesh, India beBeeEngineering Full time ₹ 9,00,000 - ₹ 11,40,000

    Backend AI Engineer JobOur mission at Elimai.ai is to empower India's key institutions by leveraging their data for solving critical issues. We develop software that comprehensively analyzes data, providing insights previously inaccessible.We are building Generative AI-native solutions that power applications across healthcare and fintech. As a Backend AI...

  • Building Scalable Cloud Platforms

    1 week ago

    Ellore, Andhra Pradesh, India beBeePlatform Full time ₹ 30,00,000 - ₹ 45,00,000

    Job OpportunityAzure Platform Engineers at Horizon56 design, build and maintain scalable, secure and reliable cloud platforms on Microsoft Azure.We are seeking an experienced engineer to work closely with development, DevOps, SRE and security teams to enable efficient delivery of applications and services while ensuring compliance, cost optimization, and...

  • Scalable Data Solutions Architect

    2 weeks ago

    Ellore, Andhra Pradesh, India beBeeData Full time ₹ 10,00,000 - ₹ 15,00,000

    About the Role:We are seeking a skilled Data Engineer to design, build and optimize scalable data solutions using Google Cloud Platform (GCP) and AtScale.This is an exciting opportunity for individuals who want to work on large-scale data projects, collaborate with cross-functional teams, and drive business growth through innovative technology solutions.Key...

  • Scalable Cloud Solutions Engineer

    7 days ago

    Ellore, Andhra Pradesh, India beBeeCloudDeveloper Full time ₹ 20,00,000 - ₹ 25,00,000

    Cloud Developer PositionWe are looking for a skilled software developer with cloud expertise to join our team.The ideal candidate will be responsible for designing, developing, and maintaining scalable cloud-based services using Python. They will also be in charge of deploying and managing applications on cloud platforms (AWS, Azure, or GCP), implementing...

  • AWS Data Engineer

    5 days ago

    Ellore, Andhra Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job DescriptionWe are seeking an accomplished Data Solutions Developer to design, develop and maintain scalable data pipelines and ETL/ELT solutions using cloud-based services.Key Responsibilities:Ingest, process and store structured/unstructured data from multiple sources (IoT devices, sensors, ERP systems, drilling logs, production data, etc.).Implement...

  • Highly Scalable Systems Architect

    7 days ago

    Ellore, Andhra Pradesh, India beBeeSystemReliability Full time ₹ 16,57,100 - ₹ 25,54,300

    Job DescriptionA highly skilled System Reliability Expert is sought to work with a leading financial services organization. The role involves managing the end-to-end application and system stack, ensuring high reliability, scalability, and performance of distributed systems.Key Responsibilities:Engage in and improve the software development lifecycle –...

  • Software Expert to Develop Scalable Systems

    3 days ago

    Ellore, Andhra Pradesh, India beBeeSoftwareExpert Full time ₹ 20,00,000 - ₹ 25,00,000

    Seeking a Software Expert to Build Innovative SolutionsWe're looking for an exceptional software builder who thrives in creating high-quality products from scratch and ensuring their scalability, security, and reliability.This role requires collaboration with clients and stakeholders to drive product direction. The ideal candidate will be passionate about...

  • A Pioneer in Scalable Data Pipelines

    1 week ago

    Ellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,29,74,400 - ₹ 2,03,60,600

    Data Engineering ProfessionalAs a skilled Data Engineer, you will play a pivotal role in designing and developing scalable data pipelines that drive business growth. Our innovative platform harnesses the power of big data to deliver real-time insights and strategic recommendations.Key Responsibilities:Design and build production-level data pipelines that...

  • Artificial Intelligence Engineer

    4 days ago

    Ellore, Andhra Pradesh, India beBeeMachineLearning Full time ₹ 25,00,000 - ₹ 48,00,000

    Are you a skilled machine learning engineer looking to take your career to the next level? This is an exciting opportunity to join our team and contribute to cutting-edge research in artificial intelligence.As a member of our team, you will work closely with researchers to develop high-performance and scalable software that addresses novel machine learning...