Building Scalable Solutions as a Data Engineer

1 day ago

Solapur, Maharashtra, India beBeeDataEngineering Full time ₹ 90,00,000 - ₹ 1,20,00,000
Job Opportunity: Data Engineering Specialist

We are seeking a skilled and innovative Data Engineering Specialist to play a critical role in shaping our technology vision, architecture, and engineering culture.

This position is ideal for someone who thrives on early-stage challenges and loves building scalable solutions from day zero.

Responsibilities:
  • Web Scraping & Crawling: Develop automated scrapers to extract structured and unstructured data from websites, APIs, and public datasets.
    • Scalable Scraping Systems: Design multi-threaded, distributed crawlers capable of handling high-volume data collection without interruptions.
      • Data Parsing & Cleaning: Normalize scraped data, remove noise, and ensure consistency before passing to data pipelines.
        • Anti-bot & Evasion Tactics: Implement proxy rotation, captcha solving, and request throttling techniques to handle scraping restrictions.
          • Integration with Pipelines: Deliver clean, structured datasets into NoSQL stores and ETL pipelines for further enrichment and graph-based storage.
            • Data Quality & Validation: Ensure data accuracy, deduplicate records, and maintain a trust scoring system for data confidence.
              • Documentation & Maintenance: Keep scrapers updated when websites change, and document scraping logic for reproducibility.
              Requirements:
              • Technical Skills:
                • 2+ years of experience in web scraping, crawling, or data collection.
                • Strong proficiency in Python (libraries like BeautifulSoup, Scrapy, Selenium, Playwright, Requests).
                • Familiarity with NoSQL databases (MongoDB, DynamoDB) and data serialization formats (JSON, CSV, Parquet).
                • Experience in handling large-scale scraping with proxy management and rate-limiting.
                • Basic knowledge of ETL processes and integration with data pipelines.
                • Exposure to graph databases (Neo4j) is a plus.
                • Soft Skills:
                  • Detail-oriented, ensuring accuracy and reliability of collected data.
                  • Strong problem-solving skills, particularly in adapting scrapers to evolving web structures.
                  • Curious mindset with a drive to discover new data sources.
                  • Comfortable working in a fast-paced, early-stage startup environment.
                  About Us:

Our company is dedicated to solving one of the most pressing business challenges - fraud detection and prevention. We're building cutting-edge, real-time products designed to empower consumers and businesses to stay ahead of fraudsters, leveraging innovative technology and deep domain expertise.

Our culture and values:

We're united by a single, critical mission - stopping fraud before it impacts businesses. Curiosity, innovation, and proactive action define our approach. We value transparency, collaboration, and individual ownership, creating an environment where talented people can do their best work.

  • Problem-Driven Innovation: We're deeply committed to solving real challenges that genuinely matter for our customers.
  • Rapid Action & Ownership: We encourage autonomy and accountability—own your projects, move quickly, and shape the future of our company.
  • Collaborative Excellence: Cross-team collaboration ensures alignment, sparks innovation, and drives us forward together.
  • Continuous Learning: Fraud evolves rapidly, and so do we. Continuous improvement, experimentation, and learning are core to our success.

  • Data Engineer – Scalable Solutions

    5 days ago

    Solapur, Maharashtra, India beBeeData Full time ₹ 22,00,000 - ₹ 25,50,000

    Job Title: Data Engineer - Scalable SolutionsOverview:We are seeking a highly motivated and experienced Data Engineer to design, build, and maintain robust data pipelines. The ideal candidate will have deep expertise in cloud-based data processing technologies using GCP, AWS, or Azure.Key Responsibilities:Design and Develop: Scalable data pipelines using...

  • Data Engineer for Scalable Data Solutions

    16 hours ago

    Solapur, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    About People Prime Worldwide:We are a technology-driven organization committed to unleashing human energy through technology for an inclusive and sustainable future. We help organizations accelerate their transition to a digital and sustainable world by providing a wide range of services, including consulting, technology, professional, and outsourcing...

  • Scalable Data Engineering Expert

    6 days ago

    Solapur, Maharashtra, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Data Pipeline SpecialistWe're seeking a seasoned Data Engineer to design, develop and optimize scalable data pipelines using AWS services.Develop efficient data ingestion, transformation and delivery processes for multiple data sources.Implement ETL/ELT processes for ensuring data quality and pipeline reliability.Optimize storage solutions using Amazon S3...

  • Senior Data Solutions Engineer

    3 hours ago

    Solapur, Maharashtra, India beBeeData Full time ₹ 8,00,000 - ₹ 16,00,000

    Job Title:Data Architect SpecialistAbout the Role:We are seeking a skilled Data Architect Specialist to lead the design and development of our data solutions.The successful candidate will have a strong background in data engineering, cloud computing, and big data technologies. They will be responsible for building scalable data pipelines, ensuring data...

  • Data Solutions Engineer

    1 week ago

    Solapur, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Our organization seeks a skilled professional to design and implement scalable, secure, and efficient data solutions.The ideal candidate will have 5+ years of experience working with Databricks, including platform architecture, data engineering, and data analytics.Key responsibilities include:Designing and implementing Databricks solutions that meet client...

  • Data Engineer

    3 weeks ago

    Solapur, Maharashtra, India CareerXperts Consulting Full time

    We are seeking a skilled Data Engineer to design, build, and optimize scalable data infrastructure that powers analytics, reporting, and machine learning. You will work closely with data scientists, analysts, and engineers to ensure data is accurate, reliable, and accessible across the organization.Key ResponsibilitiesDesign, build, and maintain data...

  • GCP Data Engineer

    3 weeks ago

    Solapur, Maharashtra, India RapidBrains Full time

    GCP Data Engineer Experience - 4+ Years Location - Remote Contract Duration - Long Term Work Time - 2:00 PM to 11:00 PMJob Description We are seeking a skilled GCP Data Engineer to design, build, and optimize scalable data pipelines and cloud-based solutions on Google Cloud Platform. The ideal candidate will have strong expertise in data engineering best...

  • Building Scalable Backend Systems

    7 hours ago

    Solapur, Maharashtra, India beBeeSoftware Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Software ArchitectWe are seeking a skilled Software Architect to design and develop scalable backend systems that deliver high-quality customer experiences.About the RoleDesign and implement core backend systems that influence how customers interact with our products.Architect and build backend services in multiple programming languages and...

  • Unlock Your Data Engineering Future

    1 day ago

    Solapur, Maharashtra, India beBeeDataEngineering Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Unlock Your Data Engineering PotentialAs a Data Engineer, you will work with cross-functional teams to design and build robust, scalable data pipelines that support analytics and reporting needs.Key Responsibilities:Data Pipeline Development: Design and develop high-quality data pipelines using Azure Data Factory, Databricks, Snowflake, and Azure SQL...

  • Menu Data Solutions Expert

    3 days ago

    Solapur, Maharashtra, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Description:">We are seeking a skilled Data Engineer to join our team in Hyderabad, India.The successful candidate will be responsible for designing and implementing scalable data solutions to support the Brand Marketing / Menu function with a specific focus on the Menu Data product and initiatives.As a Data Engineer, you will collaborate with data...