Data Crawler Systems Developer
2 weeks ago
At ELife Transfer, we're a fast-growing start-up headquartered in San Francisco, CA, USA. We recruit worldwide as our customer base is global.
We're a reliable ground transportation provider with a global reach and a diverse fleet of vehicles.
Vision: Reliable ground transportation services globally with all types of vehicles.
Mission: Empower high-quality local fleets.
Corporate Culture:
- Partner-centric
- Team collaboration
- End-to-end ownership
- Continuous learning and improvement
- Hard-working and pragmatic
- Result-driven
This role involves designing and developing web crawler systems to scrape, clean, and analyze data from various platforms. The ideal candidate will have a deep understanding of how web crawlers work, familiarity with common anti-crawling techniques and countermeasures, and the ability to handle large-scale data processing.
Primary Responsibilities:
- Design and develop efficient web crawler systems to meet business data scraping requirements.
- Conduct scraping strategy analysis on target websites and formulate optimal scraping plans.
- Maintain and optimize existing crawler systems to improve data scraping speed and accuracy.
- Clean and process scraped data to ensure data quality and availability.
- Keep track of and research the latest crawler technologies and anti-crawling mechanisms to continuously enhance the performance of crawler systems.
The ideal candidate should have at least 5 years of experience in similar positions and hold a Bachelor's degree or above in Computer Science or a related field. Proficiency in Python programming, familiarity with commonly used crawler frameworks (such as Scrapy, PySpider, etc.) and information extraction techniques (such as regular expressions) are also required.
We offer an estimated salary range of $120,000 - $180,000 per annum, depending on experience and qualifications.
Requirements:
- 5+ years experience in similar positions
- Bachelor's degree or above in Computer Science or a related field
- Proficient in Python programming and crawler frameworks
- Familiarity with information extraction techniques
- Experience in designing and developing distributed systems
- Good problem-solving skills and strong teamwork spirit
-
Data Scientist
1 week ago
Pune, Maharashtra, India ELife Full timeOur Company: A pioneering tech firm based in San Francisco, CA, USA at the heart of Silicon Valley. We recruit globally as our customer base spans worldwide. A leading provider of reliable ground transportation services globally.Vision:We aim to provide reliable ground transportation services globally with various types of vehicles.Mission:We strive to...
-
Data Scientist
4 weeks ago
Pune, Maharashtra, India NielsenIQ Full timeAbout NielsenIQNielsenIQ is the world's leading consumer intelligence company, delivering the most complete understanding of consumer buying behavior and revealing new pathways to growth.Job OverviewWe are looking for a highly skilled Data Scientist to join our team in Pune, India. As a Data Scientist, you will be responsible for developing and implementing...
-
Embedded Systems Developer
2 days ago
Pune, Maharashtra, India Innover Systems Full timeInnover Systems is seeking a highly skilled Embedded Systems Developer to join our team.We are a leading provider of IoT solutions, and we need someone with expertise in ThingsBoard and Java to help us create innovative products. The ideal candidate will have 3-6 years of professional experience in IoT development and a bachelor's degree in Computer Science...
-
Mainframe CICS Systems Developer
3 weeks ago
Pune, Maharashtra, India NTT DATA Services Full timeAbout the OpportunityWe are seeking a skilled Mainframe CICS Systems Developer to join our team in Pune, India. The successful candidate will have experience working with CICS TS v3.2 and higher, as well as automation within CICS systems duties.Key ResponsibilitiesSupport CICS release levels of TS v3.2 and higher.Upgrade CICS TS environment with new...
-
Chief Data Privacy Officer
3 weeks ago
Pune, Maharashtra, India T-Systems Full timeJob OverviewWe are seeking a seasoned Chief Data Privacy Officer to join our team at T-Systems. This is a fantastic opportunity to leverage your expertise in data privacy and security to drive business growth while ensuring compliance with regulatory requirements.Estimated Salary:€120,000 - €180,000 per annum, depending on...
-
Principal Data Architect
4 weeks ago
Pune, Maharashtra, India Data Axle Full timeAbout Data Axle:Data Axle Inc. is a 50-year-old industry leader in data, marketing solutions, sales, and research in the US.We have set up a strategic global centre of excellence in Pune, delivering mission-critical data services to our global customers powered by our proprietary cloud-based technology platform and leveraging our proprietary business &...
-
Senior Big Data Developer
4 weeks ago
Pune, Maharashtra, India NTT DATA Full timeCompany OverviewNTT DATA is a global technology services company that helps clients innovate, optimize and transform for long term success. We have diverse experts in more than 50 countries and a robust partner ecosystem of established and start-up companies.Our Commitment to YouEstimated Salary: ₹2,200,000 - ₹3,500,000 per annum (dependent on...
-
Senior Data Systems Developer
23 hours ago
Pune, Maharashtra, India ISA Full timeRole OverviewWe are seeking a highly skilled Senior Data Systems Developer to join our team at ISA, a leading technology solution provider for the aviation industry.
-
Data Storage Architect
23 hours ago
Pune, Maharashtra, India Data Dynamics Full time**Job Overview**Data Dynamics is a leading company in data management solutions. We are seeking a skilled Storage Solution Architect to join our team.The successful candidate will be responsible for designing and implementing scalable storage architectures for our clients.Key Responsibilities:Conduct technical assessments to identify customer...
-
Senior IoT Application Developer
3 weeks ago
Pune, Maharashtra, India Innover Systems Full timeInnover Systems is a cutting-edge technology company seeking an experienced Senior IoT Application Developer to spearhead data-driven innovations in our IoT solutions.Estimated Salary: ₹1,200,000 - ₹1,800,000 per annumWe are looking for a highly skilled professional with expertise in ThingsBoard and Java to design, develop, and deploy scalable IoT...
-
Agile Systems Developer
23 hours ago
Pune, Maharashtra, India Agile Soft Systems Inc Full timeJob Summary:We are seeking a skilled .NET developer to join our team at Agile Soft Systems Inc. in Pune, Nagpur & Mohali. This role involves working on challenging projects and delivering high-quality solutions using the latest technologies.About Us:Agile Soft Systems is a professional IT Services company based in Silicon Valley (Fremont, CA) with off-shore...
-
CICS System Administrator
2 weeks ago
Pune, Maharashtra, India NTT DATA Services Full timeAbout the RoleWe are seeking an experienced CICS System Programmer to join our team in Pune, India. This is a challenging role that requires strong technical skills and excellent problem-solving abilities.Key ResponsibilitiesSupport CICS release levels of TS v3.2 and higherUpgrade CICS TS environment with new releasesSupport CICSPlex SM environmentPerform...
-
Data Systems Engineer
1 month ago
Pune, Maharashtra, India Rohini IT Consulting LLP Full timeRohini IT Consulting LLP seeks a highly skilled Data Systems Engineer to design, implement, and maintain large-scale data processing systems.Key Responsibilities:- Design and develop scalable data pipelines using Star/Snowflake schema design and data modeling.- Collaborate with cross-functional teams to implement data warehouse technologies (e.g., Snowflake,...
-
Digital Data Management Specialist
23 hours ago
Pune, Maharashtra, India Data Dynamics Full time**Join Our Team**Data Dynamics is a leading company in data management solutions. We are seeking a skilled Digital Data Management Specialist to join our team.The successful candidate will be responsible for designing and implementing scalable storage architectures for our clients.Responsibilities:Conduct technical assessments to identify customer...
-
Senior Data Architect
4 weeks ago
Pune, Maharashtra, India Persistent Systems Full timeAbout Persistent SystemsPersistent Systems is a leading technology services company that enables its clients to deliver mission-critical solutions by providing innovative, reliable, and efficient technology services.Job Title: Senior Data ArchitectEstimated Salary: $120,000 - $180,000 per yearThis role requires a strong background in data engineering, with...
-
General Counsel Data Protection
23 hours ago
Pune, Maharashtra, India T-Systems Full timeAbout T-Systems:T-Systems is a leading IT services company with a strong focus on Data Privacy and Security. As a Senior Legal Advisor, you will play a crucial role in ensuring our compliance with data protection regulations.Your Key Responsibilities:Draft and review contract agreements for vendors and customersNegotiate contracts from a legal...
-
Data Systems Architect
23 hours ago
Pune, Maharashtra, India Digital Business People Full timeAbout the Job">Digital Business People is looking for an experienced Data Systems Architect to lead the design and implementation of our data infrastructure. As a key member of our team, you will be responsible for ensuring the scalability, security, and performance of our data systems.">Key Responsibilities:">Data Infrastructure Design: Design and implement...
-
Highly Skilled CICS Systems Programmer Wanted
3 weeks ago
Pune, Maharashtra, India NTT DATA Full timeAbout the RoleWe are seeking an experienced CICS System Programmer to join our team in Pune, India.Job DescriptionThe successful candidate will be responsible for supporting CICS release levels of TS v3.2 and higher, upgrading CICS TS environment with new releases, and ensuring smooth operations of CICSPlex SM environment.Main ResponsibilitiesSupport CICS...
-
Data Systems Manager
2 weeks ago
Pune, Maharashtra, India Wall Street Consulting Services LLC Full timeJob Title: Data Systems ManagerWe are seeking a highly skilled Data Systems Manager to join our team at Wall Street Consulting Services LLC in Pune/Hyderabad/Bangalore. Role Overview:As a Data Systems Manager, you will be responsible for designing, implementing and maintaining a secure SAS GRID environment. This includes managing platform security via SAS...
-
Pune, Maharashtra, India Sarvaha Systems Full timeCompany Overview:Sarvaha Systems is a niche software development company collaborating with top-funded startups and established businesses worldwide. We strive to deliver innovative solutions that meet our clients' needs.