Python Developer/ Web Scraper

4 days ago

India Hir Infotech Full time ₹ 8,00,000 - ₹ 15,00,000 per year
Web Scraping & Data Intelligence
01
Available Positions
2–3 years in web scraping, data analytics, and data-driven insights
Experience
BCA, BE.IT/CS, MCA, BSc.IT/CS or equivalent
Qualification
Development
Department
Key Responsibilities
Web Scraping & Data Acquisition

Design, develop, and maintain robust web scraping pipelines using Python frameworks such as Scrapy, Selenium, Playwright, and Beautiful Soup. Implement headless browser automation and proxy rotation to extract structured and unstructured data at scale.

Data Processing & Analysis

Build ETL/ELT workflows with pandas, NumPy, and Dask for efficient data cleansing, transformation, and normalization. Integrate API-based data sources (REST, GraphQL) and schedule batch or real-time ingestion via Apache Airflow or Prefect.

Insight Generation & Visualization

Leverage Python libraries (Matplotlib, Seaborn, Plotly) and BI tools (Tableau, Power BI) to convert raw data into actionable dashboards. Produce executive-ready reports highlighting key trends, anomalies, and opportunities.

Data Intelligence & Machine Learning

Collaborate with data science teams to deploy simple ML models (scikit-learn, XGBoost) for classification, clustering, and predictive analytics. Implement NLP pipelines (spaCy, NLTK) to extract semantics from text-based web content.

AI-Driven Automation

Explore AI-powered scraping and data enrichment services (Diffbot, with AI modules) and leverage OpenAI APIs or LangChain for intelligent data parsing, summarization, and anomaly detection.

Performance & Reliability

Ensure scraper resiliency by handling anti-bot measures, CAPTCHAs, and dynamic content. Optimize code for concurrency (Asyncio, multiprocessing) and implement unit tests (pytest) and CI/CD pipelines (GitHub Actions, Jenkins).

Proof of Impact

Demonstrate successful delivery of high-volume scraping projects—processing 5M+ records monthly with 99% data accuracy—and generation of insights that led to a 25% improvement in decision-making speed.

Required Qualifications
  • Bachelor's degree in Computer Science, Data Science, or related field with fluency in English communication
  • Proficiency in Python, web frameworks (Flask, FastAPI), and SQL/noSQL databases (PostgreSQL, MongoDB)
  • Familiarity with cloud environments (AWS Lambda, EC2, S3; Azure Functions) and containerization (Docker, Kubernetes)
  • Strong problem-solving skills, attention to detail, and ability to translate complex data challenges into scalable solutions

Preferred: Experience with graph databases (Neo4j), vector databases (Pinecone), and real-time streaming (Kafka, AWS Kinesis).

  • Senior Python Developer

    4 weeks ago

    India CJ Konsultants Full time

    Position: Python Developer Location: Remote (India) Salary: 80-90k pm Experience: 4+ years Job Description: We’re looking for a Python Developer with solid hands-on experience in job and web data scraping, AI-enabled data processing, and building scalable ETL pipelines. You’ll design, implement, and operate scraping and enrichment systems that feed...

  • Python Developer Intern

    3 weeks ago

    Delhi, India Bloom AI Full time

    Job Description About Bloom AI Bloom AI is a modern intelligence firm that accelerates decision-making through AI-driven synthesized intelligence. We empower enterprises to unlock the value of data with human-like synthesis and decision intelligence at scale. Our proprietary tools and solutions are trusted by investment managers, insurance, private equity,...

  • Senior Python Developer

    4 weeks ago

    India CJ Konsultants Full time

    Position: Python DeveloperLocation: Remote (India)Salary: 80-90k pmExperience: 4+ yearsJob Description: We’re looking for a Python Developer with solid hands-on experience in job and web data scraping, AI-enabled data processing, and building scalable ETL pipelines. You’ll design, implement, and operate scraping and enrichment systems that feed...

  • Data Engineer

    3 weeks ago

    India Alternative Path Full time

    Alternative Path is seeking skilled software developers to collaborate on client projects with an asset management firm. In this role, you will collaborate with individuals across various company departments to shape and innovate new products and features for our platform, enhancing existing ones. You will have a large degree of independence and trust, but...

  • Data Engineer

    2 weeks ago

    India Alternative Path Full time

    Alternative Path is seeking skilled software developers to collaborate on client projects with an asset management firm. In this role, you will collaborate with individuals across various company departments to shape and innovate new products and features for our platform, enhancing existing ones. You will have a large degree of independence and trust, but...

  • Data Engineer

    3 weeks ago

    India Alternative Path Full time

    Alternative Path is seeking skilled software developers to collaborate on client projects with an asset management firm. In this role, you will collaborate with individuals across various company departments to shape and innovate new products and features for our platform, enhancing existing ones. You will have a large degree of independence and trust, but...

  • Data Engineer

    3 weeks ago

    India Alternative Path Full time

    Alternative Path is seeking skilled software developers to collaborate on client projects with an asset management firm. In this role, you will collaborate with individuals across various company departments to shape and innovate new products and features for our platform, enhancing existing ones. You will have a large degree of independence and trust, but...

  • Data Engineer

    3 weeks ago

    India Taiyō.AI Full time

    About Us Taiyo.AI is the world's first infrastructure intelligence platform. We are building the largest universal and industry standard database of opportunities (tenders, projects, news) and threats (economy, climate, geopolitics, finance, logistics, etc.) for real assets. Taiyo.AI has been instrumental in shaping how infrastructure companies (infra...

  • Data Engineer

    3 weeks ago

    India Taiyō.AI Full time

    About Us Taiyo.AI is the world's first infrastructure intelligence platform. We are building the largest universal and industry standard database of opportunities (tenders, projects, news) and threats (economy, climate, geopolitics, finance, logistics, etc.) for real assets. Taiyo.AI has been instrumental in shaping how infrastructure companies (infra...

  • Automations / Product Engineer

    3 days ago

    India Activate Talent Full time

    Job Title Automations Product Engineer Job Type Full-time Remote Schedule US Time zone Industry Creator Affiliate Marketing Software About the Role We re looking for an Automations Product Engineer to own our Discovery area the place where brands find and connect with creators and ambassadors You ll be responsible for the data backbone behind that experience...