eMedEvents - Python Developer - Web Scraping

2 weeks ago

Hyderabad, India eMedEvents Full time

Python Developer Web Scraping & Data Processing

Experience : 3+ Years

Employment Type : Full-time

Job Overview :

We are seeking a skilled and detail-oriented Python Developer with 3+ years of hands-on experience in web scraping, document parsing (PDF, HTML, XML), and structured data extraction. You will be a vital part of a core team focused on aggregating biomedical content from diverse sources, including grant repositories, scientific journals, conference abstracts, treatment guidelines, and clinical trial databases. This role demands strong technical proficiency in various parsing and scraping libraries, along with solid data processing and integration skills.

Key Responsibilities :

- Develop scalable Python scripts to effectively scrape and parse biomedical data from a wide range of web sources, including websites, pre-print servers, citation indexes, scientific journals, and treatment guidelines.

- Build robust modules specifically for splitting multi-record documents (such as PDFs, HTML, and other formats) into individual, manageable content units.

- Implement NLP-based field extraction pipelines utilizing libraries like spaCy, NLTK, or advanced regex for precise metadata tagging.

- Design and automate complex data acquisition workflows using schedulers and orchestrators like cron, Celery, or Apache Airflow for periodic scraping and content updates.

- Store parsed and processed data efficiently in both relational (PostgreSQL) and NoSQL (MongoDB) databases, ensuring optimal schema design for performance and scalability.

- Ensure robust logging, comprehensive exception handling, and rigorous content quality validation across all data processing and scraping workflows.

Required Skills and Qualifications :

- 3+ years of hands-on experience in Python, particularly focused on data extraction, transformation, and loading (ETL).

- Strong command over web scraping libraries, including :

1. BeautifulSoup

2. Scrapy

3. Selenium

4. Playwright

- Proficiency in PDF parsing libraries, such as :

1. PyMuPDF

2. pdfminer.six

3. PDFPlumber

- Experience with HTML/XML parsers: lxml, XPath, html5lib.

- Familiarity with regular expressions, NLP concepts, and advanced field extraction techniques.

- Working knowledge of SQL and/or NoSQL databases (MySQL, PostgreSQL, MongoDB).

- Understanding of API integration (RESTful APIs) for interacting with structured data sources.

- Experience with task schedulers and workflow orchestrators (cron, Apache Airflow, Celery).

- Proficiency in version control using Git/GitHub and comfort working in collaborative development environments.

Good to Have :

- Exposure to biomedical or healthcare data parsing (scientific abstracts, clinical trials data, drug labels).

- Familiarity with cloud environments like AWS (specifically Lambda, S3 for data storage and processing).

- Experience with data validation frameworks and building robust QA rules for data quality.

- Understanding of ontologies and taxonomies (UMLS, MeSH) for structured content tagging.


(ref:hirist.tech)

  • Senior web scraping engineer

    5 days ago

    Hyderabad, India S2T AI - AI-Powered Investigations Full time

    We are on the lookout for a highly competent, self-motivated Senior Web Scraping Engineer with real-world experience in API Scraping or Mobile Scraping to join our India team.The Role:Gather and process raw data at scale (including writing scripts, web scraping, calling APIs)Work in a larger team.Able to work independently to complete assigned tasksThe...

  • Business Development Manager

    2 weeks ago

    Hyderabad, India eMedEvents - Global Marketplace for CMECE Full time

    Job Title: Sales Representative (Marketplace and Marketing Services) Company Overview: eMedEvents is a pioneering platform dedicated to revolutionizing medical education. Our online marketplace connects healthcare professionals with a diverse range of continuing medical education (CME) events. As part of our commitment to enhancing the experience for event...

  • Business Development Manager

    4 days ago

    Hyderabad, Telangana, India eMedEvents - Global Marketplace for CMECE Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Job Title: Sales Representative (Marketplace and Marketing Services)Company Overview: eMedEvents is a pioneering platform dedicated to revolutionizing medical education. Our online marketplace connects healthcare professionals with a diverse range of continuing medical education (CME) events. As part of our commitment to enhancing the experience for event...

  • Business Development Manager

    1 week ago

    Hyderabad, India eMedEvents - Global Marketplace for CMECE Full time

    Job Description Job Title: Sales Representative (Marketplace and Marketing Services) Company Overview: eMedEvents is a pioneering platform dedicated to revolutionizing medical education. Our online marketplace connects healthcare professionals with a diverse range of continuing medical education (CME) events. As part of our commitment to enhancing the...

  • Business Development Manager

    2 days ago

    hyderabad, India eMedEvents - Global Marketplace for CMECE Full time

    Job Title: Sales Representative (Marketplace and Marketing Services) Company Overview: eMedEvents is a pioneering platform dedicated to revolutionizing medical education. Our online marketplace connects healthcare professionals with a diverse range of continuing medical education (CME) events. As part of our commitment to enhancing the experience for event...

  • Business Development Manager

    2 days ago

    Hyderabad, India eMedEvents - Global Marketplace for CMECE Full time

    Job Title: Sales Representative (Marketplace and Marketing Services) Company Overview: eMedEvents is a pioneering platform dedicated to revolutionizing medical education. Our online marketplace connects healthcare professionals with a diverse range of continuing medical education (CME) events. As part of our commitment to enhancing the experience for event...

  • Business Development Manager

    3 days ago

    Hyderabad, India eMedEvents - Global Marketplace for CMECE Full time

    Job Title: Sales Representative (Marketplace and Marketing Services) Company Overview: eMedEvents is a pioneering platform dedicated to revolutionizing medical education. Our online marketplace connects healthcare professionals with a diverse range of continuing medical education (CME) events. As part of our commitment to enhancing the experience for event...

  • Python Developer

    3 weeks ago

    Hyderabad, Telangana, India Akshaya IT Business solutions Full time

    Python Web Crawling DeveloperWe are seeking a highly skilled and motivated Python Web Crawling Developer with 5 to 10+ years of hands-on experience in web scraping and data extraction. The ideal candidate should have a solid background in Python-based scraping tools and libraries, and a proven track record of working on dynamic websites.Responsibilities :-...

  • Senior JavaScript Developer(Web Scraping)

    24 hours ago

    HITEC City, Hyderabad, Telangana, India Interaslabs Full time ₹ 3,65,000 - ₹ 35,82,285 per year

    Senior JavaScript Developer (Web Scraping)Location: Hyderabad (Hybrid)Experience: 6+ YearsEmployment Type: Full-timeAbout the RoleWe are looking for a Senior JavaScript Developer with strong expertise in web scraping and backend systems. The ideal candidate will design and maintain scalable scraping pipelines, work on distributed backend systems, and deploy...

  • Sr Developer(Web Scraping)

    2 weeks ago

    Hyderabad, India Interaslabs Full time

    Job Title: Senior Developer-Javascript Web Scraping Location: Hyderabad Type: Full-time Experience: 10+ years About the Role: This role involves building and maintaining large-scale, data-heavy applications that process terabytes of data daily. You will be responsible for designing scalable backend solutions and working on real-time data processing, API...