Data Scraping Solutions Architect

14 hours ago

Kolkata, West Bengal, India beBeeArchitect Full time ₹ 25,00,000 - ₹ 30,00,000

Job Summary:

We are seeking an experienced Data Scraping Solutions Architect to drive our data extraction initiatives forward. The ideal candidate will have a minimum of 4 years of hands-on experience in IT scraping, with at least 2 years leading a team of 5+ developers.


This role requires deep technical knowledge in advanced scraping techniques, reverse engineering, automation, and leadership skills to drive the team towards success.

Main Responsibilities:

  • Design and develop scalable data scraping solutions using tools like Scrapy and Python libraries.
  • Lead and mentor a team of 5+ developers, managing project timelines and deliverables.
  • Implement advanced blocking and captcha-solving techniques to bypass scraping restrictions.
  • Conduct source code reverse engineering and automate web and app interactions.
  • Manage proxies, IP rotation, and SSL unpinning to ensure effective scraping.
  • Maintain and improve API integrations and data pipelines.
  • Ensure code quality through effective version control, error handling, and documentation.
  • Collaborate with cross-functional teams for project planning and execution.
  • Monitor performance and provide solutions under high-pressure environments.

Required Skills and Qualifications:

  • Data Scraping: Minimum 4 years in IT scraping industry
  • Leadership: Minimum 2 years leading a team of 5+ developers
  • Scraping Tools: Scrapy, Threading, requests, web automation
  • Technical Proficiency:
    • Advanced Python
    • Captcha solving and blocking handling
    • Source reverse engineering
    • Proxy management & IP rotation
    • App automation, SSL Unpin, Frida
    • API Management, Version Control Systems
    • Error Handling, SQL, MongoDB, Pandas
  • Leadership Skills:
    • Project management
    • Documentation
    • Team handling
    • Pressure management
    • Flexibility and adaptability
    • Accountability

Preferred (Good to Have):

  • Experience with Linux
  • Knowledge of Appium, Fiddler, Burp Suite

  • Solution Architect

    6 hours ago

    Kolkata, West Bengal, India BayOne Solutions Full time

    Role: Solution Architect (AI Infrastructure, Kubernetes & HPC)Location: Bangalore/PuneDuration: Full Time"Just a heads-up — the second round will need to be attended in person, as per the client's request."Role Type: Senior Role - Delivery-focused infrastructure role (non–presales; may include limited presales engagement)Core Focus AreasKubernetes...

  • Data Architect

    4 weeks ago

    Kolkata, West Bengal, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    Job Title : Data ArchitectExperience : 913 yearsLocation : Bangalore, Pune, Gurgaon, Kolkata, Hyderabad, ChennaiJob Description : We are looking for an experienced Data Architect with expertise in Azure Cloud, Databricks. The ideal candidate should have strong skills in solution design, data architecture, and end-to-end data solutioning to drive scalable...

  • Senior Solution Architect

    13 hours ago

    Kolkata, West Bengal, India Esko Full time

    Job DescriptionWe are seeking a Senior Solution Architect to lead the end-to-end design and implementation of enterprise software. This role is a perfect fit for an experienced professional with a strong development background who can provide thought leadership and technical guidance to both internal teams and customers. You will work on a hybrid model, with...

  • Chief Data Architect

    2 days ago

    Kolkata, West Bengal, India beBeeDatabase Full time

    Database Architect Job DescriptionAbout the Role:The Database Architect will be responsible for designing and implementing database solutions that meet the needs of our organization.Key Responsibilities:Design and implement database architectures that are scalable, secure, and efficient.Develop and maintain database schema, including table relationships and...

  • AWS Solution Architect

    3 weeks ago

    Kolkata, West Bengal, India LTIMindtree Limited Full time

    Job DescriptionJob Summary:15+ years of hands on IT experience in design and development of complex systemMinimum of 5+ years in a solution or technical architect role using service and hosting solutions such as private/public cloud IaaS, PaaS and SaaS platformsAt least 4+ years of experience hands on experience in cloud native architecture design,...

  • Sify - Solution Architect - Cloud

    3 weeks ago

    Kolkata, West Bengal, India Sify Limited Full time

    - We're looking for a highly skilled and customer-focused Solution Architect, Cloud to join our growing team in India. In this critical role, you'll work closely with our Sales and Go-to-Market (GTM) teams, acting as a technical expert to design compelling cloud solutions for our customers across various industries.- Your ability to translate complex...

  • Cloud Solutions Architect

    1 day ago

    Kolkata, West Bengal, India beBeeCloudArchitect Full time US$ 1,50,000 - US$ 2,00,000

    Job Title: Cloud Solutions ArchitectWe are seeking an experienced Cloud Solutions Architect to join our team.About the RoleThis is a critical position responsible for designing, implementing, and maintaining complex cloud-based systems. The ideal candidate will have a strong understanding of cloud computing architectures, migration strategies, and security...

  • Data Architect

    1 week ago

    Kolkata, West Bengal, India Tata Consultancy Services Full time

    TCS present an excellent opportunity for Data architect Job Description: Skills: AWS, Glue, Redshift, PySpark Location: Pune / Kolkata Experience: 10 to 12 Years Strong hands-on experience in Python programming and PySpark. Experience using AWS services (RedShift, Glue, EMR, S3 & Lambda) Experience working with Apache Spark and Hadoop ecosystem. ...

  • Cloud Native Solution Architect

    4 weeks ago

    Kolkata, West Bengal, India LTIMindtree Full time

    Skills : Application development frameworks and cloud-native patterns ,Solution ArchitectExperience : 12 - 22 YearsLocation : KolkataJob Summary:Solution Architecture & DesignDesign and architect complex cloud-native solutions for enterprise clients across public and private cloud platformsDefine microservices architecture patterns and establish design...

  • Senior Data Solutions Architect

    4 days ago

    Kolkata, West Bengal, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 1,99,99,999

    Role OverviewTransform data into actionable insights to drive business growth as a seasoned data engineer. Our team focuses on designing and developing robust data solutions using advanced technologies.The ideal candidate will possess strong proficiency in Azure Databricks, including Spark and Delta Lake, along with experience in Azure Data Factory, Azure...