Manager Data Engineer
2 weeks ago
Alternative Path is seeking an experienced Technical Manager to engage and mentor a team of skilled Data engineers who collaborate on client projects with an asset management firm. In this role, you will collaborate with individuals across the team and various company departments to shape and innovate new products and features for our platform, enhancing existing ones. You will have a large degree of independence and trust, accompanied bya team of Engineers.
This is an opportunity to join a team-first meritocracy and help grow an entrepreneurial group inside Alternative Path. You will be asked to contribute, given ownership, and will be expected to make your voice heard. Role SummaryLeading and mentoring a team of seasoned Data engineers performing Web Scraping using various scraping techniques and then utilizing Python's Pandas library for data cleaning and manipulation.
Then, ingesting the data into a Warehouse, and scheduling the scrapers using Airflow or other tools. Role Overview The Web Scraping Team at Alternative Path is seeking creative and detail-oriented Leaders to contribute to client projects and lead by example.
This team develops essential applications, datasets, and alerts that directly support client investment decisions. Our focus is to maintain operational excellence by providing high-quality proprietary datasets, timely notifications, and exceptional service. The ideal candidate will be self-motivated, self-sufficient, and possess a passion for tinkering and a love for automation.
What we need ➢ Technical Leadership
- Oversee the design and implementation of web scraping projects to ensure scalability, efficiency, and accuracy.
- Stay updated with and implement the latest technologies, tools, and frameworks in web scraping and data processing.
- Review and approve pull requests to ensure clean, maintainable, and efficient code.
- Identify and solve complex technical challenges in data extraction, handling, and storage.
- Design and implement monitoring tools and dashboards to ensure system reliability and performance.
- 6 to 12 years of experience in the Data Engineering or Web Scraping industry, with a strong background in data processing and automation.
- 2+ years of experience in leading teams, mentoring engineers, and driving technical initiatives➢ Team Management (Techno-Managerial Role)
- Lead, mentor, and inspire a team of engineers to achieve project goals and professional growth.
- Conduct regular one-on-ones to provide feedback, set objectives, and discuss career development.
- Monitor team productivity and allocate resources effectively to meet deadlines.
- Foster a collaborative and high-performing team culture that promotes innovation and ownership.
- Serve in a techno-managerial role, balancing technical expertise with leadership responsibilities to drive both project execution and team development.
➢ Strategic Planning and Execution
- Collaborate with stakeholders to understand project requirements and translate them into actionable plans.
- Develop long-term strategies for web scraping solutions, including data storage, compliance, and scalability.
- Drive continuous improvements in process, tools, and methodologies for the team.
- Align team efforts with business goals and identify opportunities for automation and process optimization.
➢ Operational Excellence
- Ensure high-quality deliverables by enforcing best practices in coding, testing, and documentation.
- Optimize the data pipeline to handle large-scale data ingestion and cleaning.
- Implement safeguards for compliance with web scraping regulations and ethical practices.
- Ensure the team adopts the best practices for handling sensitive and proprietary data.
Technical Skills Required ➢ Proficiency in Python and web scraping skills is required. ➢ Must have strong expertise in using Panda's library (Python). ➢ Experience with web technologies (HTML/JS, APIs, etc.) is essential.
➢ Should have a good understanding of tools such as Scrapy, BeautifulSoup, and Selenium. ➢ Responsible for reviewing and approving pull requests to ensure clean, maintainable, and efficient code. ➢ Experience building scalable scraping solutions for large-scale data collection ➢ Familiarity with AWS technologies like S3, RDS, SNS, SQS, Lambda, and others is necessary.
-
Data Engineer
4 days ago
India NTT DATA Full time ₹ 9,00,000 - ₹ 12,00,000 per yearReq ID: 343254NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Remote, Karnātaka (IN-KA), India (IN). "Key Responsibilities: Design and implement...
-
Data engineer
5 days ago
India Data-Hat AI Full timeDepartment: Data Engineering & AI Solutions Reports To: Lead Data Solutions Architect Travel: International travel required (up to 30–40%) Position Summary: We are hiring a senior-level Data Engineer to lead the design, development, and optimization of high-performance data infrastructure that underpins mission-critical AI systems. With 12+...
-
Data Engineer
5 days ago
India Data-Hat AI Full timeDepartment: Data Engineering & AI Solutions Reports To: Lead Data Solutions Architect Travel: International travel required (up to 30–40%) Position Summary: We are hiring a senior-level Data Engineer to lead the design, development, and optimization of high-performance data infrastructure that underpins mission-critical AI systems. With 12+...
-
Data Engineer
6 days ago
India Data-Hat AI Full timeDepartment: Data Engineering & AI Solutions Reports To: Lead Data Solutions Architect Travel: International travel required (up to 30–40%) Position Summary: We are hiring a senior-level Data Engineer to lead the design, development, and optimization of high-performance data infrastructure that underpins mission-critical AI systems. With 12+ years of...
-
Data Engineer
3 days ago
india, IN Data-Hat AI Full timeDepartment: Data Engineering & AI Solutions Reports To: Lead Data Solutions Architect Travel: International travel required (up to 30–40%) Position Summary: We are hiring a senior-level Data Engineer to lead the design, development, and optimization of high-performance data infrastructure that underpins mission-critical AI systems. With 12+ years of...
-
Data Engineer
4 days ago
India Data-Hat AI Full timeDepartment: Data Engineering & AI Solutions Reports To: Lead Data Solutions Architect Travel: International travel required (up to 30–40%) Position Summary: We are hiring a senior-level Data Engineer to lead the design, development, and optimization of high-performance data infrastructure that underpins mission-critical AI systems....
-
Senior Engineer, Data Engineering
4 days ago
India Data-Hat AI Full timeDepartment: Data Engineering & AI Solutions Reports To: Lead Data Solutions Architect Travel: International travel required (up to 30–40%) We are hiring a senior-level Data Engineer to lead the design, development, and optimization of high-performance data infrastructure that underpins mission-critical AI systems. With 12+ years of experience,...
-
Lead Data Engineer
3 weeks ago
India Eucloid Data Solutions Full timeEucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and building of...
-
Lead Data Engineer
3 weeks ago
India Eucloid Data Solutions Full timeEucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and building of...
-
Lead Data Engineer
2 weeks ago
India Eucloid Data Solutions Full timeEucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and building of...