Data Crawler Systems Developer

2 weeks ago


Pune, Maharashtra, India ELife Transfer Full time

At ELife Transfer, we're a fast-growing start-up headquartered in San Francisco, CA, USA. We recruit worldwide as our customer base is global.

We're a reliable ground transportation provider with a global reach and a diverse fleet of vehicles.

Vision: Reliable ground transportation services globally with all types of vehicles.

Mission: Empower high-quality local fleets.

Corporate Culture:

  • Partner-centric
  • Team collaboration
  • End-to-end ownership
  • Continuous learning and improvement
  • Hard-working and pragmatic
  • Result-driven
Job Title: Data Crawler Systems Developer

This role involves designing and developing web crawler systems to scrape, clean, and analyze data from various platforms. The ideal candidate will have a deep understanding of how web crawlers work, familiarity with common anti-crawling techniques and countermeasures, and the ability to handle large-scale data processing.

Primary Responsibilities:

  • Design and develop efficient web crawler systems to meet business data scraping requirements.
  • Conduct scraping strategy analysis on target websites and formulate optimal scraping plans.
  • Maintain and optimize existing crawler systems to improve data scraping speed and accuracy.
  • Clean and process scraped data to ensure data quality and availability.
  • Keep track of and research the latest crawler technologies and anti-crawling mechanisms to continuously enhance the performance of crawler systems.

The ideal candidate should have at least 5 years of experience in similar positions and hold a Bachelor's degree or above in Computer Science or a related field. Proficiency in Python programming, familiarity with commonly used crawler frameworks (such as Scrapy, PySpider, etc.) and information extraction techniques (such as regular expressions) are also required.

We offer an estimated salary range of $120,000 - $180,000 per annum, depending on experience and qualifications.

Requirements:

  • 5+ years experience in similar positions
  • Bachelor's degree or above in Computer Science or a related field
  • Proficient in Python programming and crawler frameworks
  • Familiarity with information extraction techniques
  • Experience in designing and developing distributed systems
  • Good problem-solving skills and strong teamwork spirit

  • Data Scientist

    1 week ago


    Pune, Maharashtra, India ELife Full time

    Our Company: A pioneering tech firm based in San Francisco, CA, USA at the heart of Silicon Valley. We recruit globally as our customer base spans worldwide. A leading provider of reliable ground transportation services globally.Vision:We aim to provide reliable ground transportation services globally with various types of vehicles.Mission:We strive to...

  • Data Scientist

    4 weeks ago


    Pune, Maharashtra, India NielsenIQ Full time

    About NielsenIQNielsenIQ is the world's leading consumer intelligence company, delivering the most complete understanding of consumer buying behavior and revealing new pathways to growth.Job OverviewWe are looking for a highly skilled Data Scientist to join our team in Pune, India. As a Data Scientist, you will be responsible for developing and implementing...


  • Pune, Maharashtra, India Innover Systems Full time

    Innover Systems is seeking a highly skilled Embedded Systems Developer to join our team.We are a leading provider of IoT solutions, and we need someone with expertise in ThingsBoard and Java to help us create innovative products. The ideal candidate will have 3-6 years of professional experience in IoT development and a bachelor's degree in Computer Science...


  • Pune, Maharashtra, India NTT DATA Services Full time

    About the OpportunityWe are seeking a skilled Mainframe CICS Systems Developer to join our team in Pune, India. The successful candidate will have experience working with CICS TS v3.2 and higher, as well as automation within CICS systems duties.Key ResponsibilitiesSupport CICS release levels of TS v3.2 and higher.Upgrade CICS TS environment with new...


  • Pune, Maharashtra, India T-Systems Full time

    Job OverviewWe are seeking a seasoned Chief Data Privacy Officer to join our team at T-Systems. This is a fantastic opportunity to leverage your expertise in data privacy and security to drive business growth while ensuring compliance with regulatory requirements.Estimated Salary:€120,000 - €180,000 per annum, depending on...


  • Pune, Maharashtra, India Data Axle Full time

    About Data Axle:Data Axle Inc. is a 50-year-old industry leader in data, marketing solutions, sales, and research in the US.We have set up a strategic global centre of excellence in Pune, delivering mission-critical data services to our global customers powered by our proprietary cloud-based technology platform and leveraging our proprietary business &...


  • Pune, Maharashtra, India NTT DATA Full time

    Company OverviewNTT DATA is a global technology services company that helps clients innovate, optimize and transform for long term success. We have diverse experts in more than 50 countries and a robust partner ecosystem of established and start-up companies.Our Commitment to YouEstimated Salary: ₹2,200,000 - ₹3,500,000 per annum (dependent on...


  • Pune, Maharashtra, India ISA Full time

    Role OverviewWe are seeking a highly skilled Senior Data Systems Developer to join our team at ISA, a leading technology solution provider for the aviation industry.

  • Data Storage Architect

    23 hours ago


    Pune, Maharashtra, India Data Dynamics Full time

    **Job Overview**Data Dynamics is a leading company in data management solutions. We are seeking a skilled Storage Solution Architect to join our team.The successful candidate will be responsible for designing and implementing scalable storage architectures for our clients.Key Responsibilities:Conduct technical assessments to identify customer...


  • Pune, Maharashtra, India Innover Systems Full time

    Innover Systems is a cutting-edge technology company seeking an experienced Senior IoT Application Developer to spearhead data-driven innovations in our IoT solutions.Estimated Salary: ₹1,200,000 - ₹1,800,000 per annumWe are looking for a highly skilled professional with expertise in ThingsBoard and Java to design, develop, and deploy scalable IoT...


  • Pune, Maharashtra, India Agile Soft Systems Inc Full time

    Job Summary:We are seeking a skilled .NET developer to join our team at Agile Soft Systems Inc. in Pune, Nagpur & Mohali. This role involves working on challenging projects and delivering high-quality solutions using the latest technologies.About Us:Agile Soft Systems is a professional IT Services company based in Silicon Valley (Fremont, CA) with off-shore...


  • Pune, Maharashtra, India NTT DATA Services Full time

    About the RoleWe are seeking an experienced CICS System Programmer to join our team in Pune, India. This is a challenging role that requires strong technical skills and excellent problem-solving abilities.Key ResponsibilitiesSupport CICS release levels of TS v3.2 and higherUpgrade CICS TS environment with new releasesSupport CICSPlex SM environmentPerform...

  • Data Systems Engineer

    1 month ago


    Pune, Maharashtra, India Rohini IT Consulting LLP Full time

    Rohini IT Consulting LLP seeks a highly skilled Data Systems Engineer to design, implement, and maintain large-scale data processing systems.Key Responsibilities:- Design and develop scalable data pipelines using Star/Snowflake schema design and data modeling.- Collaborate with cross-functional teams to implement data warehouse technologies (e.g., Snowflake,...


  • Pune, Maharashtra, India Data Dynamics Full time

    **Join Our Team**Data Dynamics is a leading company in data management solutions. We are seeking a skilled Digital Data Management Specialist to join our team.The successful candidate will be responsible for designing and implementing scalable storage architectures for our clients.Responsibilities:Conduct technical assessments to identify customer...

  • Senior Data Architect

    4 weeks ago


    Pune, Maharashtra, India Persistent Systems Full time

    About Persistent SystemsPersistent Systems is a leading technology services company that enables its clients to deliver mission-critical solutions by providing innovative, reliable, and efficient technology services.Job Title: Senior Data ArchitectEstimated Salary: $120,000 - $180,000 per yearThis role requires a strong background in data engineering, with...


  • Pune, Maharashtra, India T-Systems Full time

    About T-Systems:T-Systems is a leading IT services company with a strong focus on Data Privacy and Security. As a Senior Legal Advisor, you will play a crucial role in ensuring our compliance with data protection regulations.Your Key Responsibilities:Draft and review contract agreements for vendors and customersNegotiate contracts from a legal...

  • Data Systems Architect

    23 hours ago


    Pune, Maharashtra, India Digital Business People Full time

    About the Job">Digital Business People is looking for an experienced Data Systems Architect to lead the design and implementation of our data infrastructure. As a key member of our team, you will be responsible for ensuring the scalability, security, and performance of our data systems.">Key Responsibilities:">Data Infrastructure Design: Design and implement...


  • Pune, Maharashtra, India NTT DATA Full time

    About the RoleWe are seeking an experienced CICS System Programmer to join our team in Pune, India.Job DescriptionThe successful candidate will be responsible for supporting CICS release levels of TS v3.2 and higher, upgrading CICS TS environment with new releases, and ensuring smooth operations of CICSPlex SM environment.Main ResponsibilitiesSupport CICS...

  • Data Systems Manager

    2 weeks ago


    Pune, Maharashtra, India Wall Street Consulting Services LLC Full time

    Job Title: Data Systems ManagerWe are seeking a highly skilled Data Systems Manager to join our team at Wall Street Consulting Services LLC in Pune/Hyderabad/Bangalore. Role Overview:As a Data Systems Manager, you will be responsible for designing, implementing and maintaining a secure SAS GRID environment. This includes managing platform security via SAS...


  • Pune, Maharashtra, India Sarvaha Systems Full time

    Company Overview:Sarvaha Systems is a niche software development company collaborating with top-funded startups and established businesses worldwide. We strive to deliver innovative solutions that meet our clients' needs.