Senior Web Crawling Data Engineer
7 days ago
Job OverviewWe are seeking an experienced data engineering leader to spearhead our web crawling and pipeline architecture initiatives.The successful candidate will be responsible for leading and mentoring data engineering and web crawling teams, ensuring high-quality delivery and adherence to best practices.They will design, implement, and optimize scalable data pipelines that support high-volume data ingestion, transformation, and storage.A strong understanding of modern web crawling frameworks and IP rotation techniques is essential.Experience with distributed processing technologies such as Airflow or Celery is required.The ideal candidate will also have expertise in defining and enforcing data quality, validation, and security measures across all data flows and pipelines.Collaboration with cross-functional teams to translate data requirements into technical solutions is a key aspect of this role.Additionally, the candidate should have hands-on experience with cloud-based deployments on AWS/GCP/Azure and AI/LLM frameworks.RequirementsBachelor's or master's degree in engineering, computer science, or related field.At least 7–12 years of relevant experience in data engineering, pipeline design, or large-scale web crawling systems.Strong expertise in Python, SQL, and modern data processing practices.Proficiency in proxy systems, anti-bot techniques, and scalable crawler architecture is necessary.Hands-on experience with cloud data platforms and AI/LLM frameworks is required.BenefitsOpportunity to work with cutting-edge technology and innovative projects.Collaborative and dynamic work environment.Professional growth and development opportunities.
-
Web Crawling Specialist
7 days ago
rajahmundry, India beBeeEngineer Full timeWeb Crawling SpecialistWe are seeking a skilled web crawling specialist to join our team. In this role, you will be responsible for designing and implementing web crawlers, extracting valuable insights from the web, and ensuring data quality.
-
Web Crawler Specialist
4 days ago
rajahmundry, India beBeeCrawling Full timeJob TitleWe are seeking a seasoned professional to take up the role of Web Crawler Specialist. The ideal candidate will be responsible for designing, developing and maintaining web crawlers that can effectively extract valuable insights from the web.Maintain and enhance existing web scraping projects.Develop and refine crawlers using Python-based tools and...
-
Data Engineering Specialist
4 days ago
rajahmundry, India beBeeDataEngineering Full timeJob Title: Data Engineering SpecialistWe are seeking a highly skilled data engineering specialist to join our team. The ideal candidate will have hands-on experience with data/content crawling, bot detection avoidance techniques, and the use of VPNs.Main Responsibilities:Designing and implementing efficient ETL processes using cloud-native...
-
Senior Web Automation Specialist
22 hours ago
rajahmundry, India beBeeAutomation Full timeJob OpportunityWe are looking for an Associate Automation Engineer to fill this role. As an entry-level position, you will gain expertise in Appium Pro and enjoy building web UIs.Write automation code for mobile applicationsCapture data and complete analysis on mobile applicationsCreate a dashboard with the captured data and provide it to customersAssist...
-
Senior Data Engineering Professional
3 days ago
rajahmundry, India beBeeData Full timeJob OpportunityWe seek an experienced Senior Data Engineer to lead the design, development, and implementation of robust data solutions on Microsoft Azure.Key Responsibilities:Data Engineering Expertise: Proven experience with SQL databases (e.g., Azure SQL/SQL Server), including query optimization and performance tuning.Azure Services: Advanced expertise in...
-
Senior Data Engineering Specialist
1 day ago
rajahmundry, India beBeeDataEngineer Full timeJob DescriptionThe ideal candidate should have hands-on coding experience and expertise in data engineering, including modeling and managing multi-dimensional entities and graphs.They must also have knowledge of graph databases and data lake technologies, along with experience in architecting and implementing ETL processes using cloud-native...
-
rajahmundry, India beBeeEngineer Full timeJob OverviewThe Director of Frontend Engineering leads a high-performing team focused on building scalable and efficient web applications.Key Responsibilities:Own and evolve the frontend architecture and component strategy to drive business growth.Manage a team of 10+ frontend engineers and managers to deliver high-quality React/Next.js applications.Develop...
-
Web Fabric Innovator
4 days ago
rajahmundry, India beBeeFrontend Full timeWe are seeking a visionary leader to spearhead our Web Fabric initiative. As Principal Frontend Engineer, you will define the architecture, scale teams, and ensure low-latency, high-quality user experience across all products.This leader will manage a team of 10+ engineers and managers, aligning with Data Engineering and ML/API teams to deliver end-to-end...
-
Senior Data Engineer
7 days ago
rajahmundry, India beBeeDataEngineer Full timeThe following job description is for a Senior Data Engineer with expertise in Databricks. About the RoleWe are seeking an experienced Data Engineer to support our enterprise-wide Sustainability initiative. The ideal candidate will have extensive knowledge of ETL/ELT pipelines on Databricks using PySpark and SQL, as well as experience working with data...
-
Senior Data Engineer
3 weeks ago
Rajahmundry, India Live Connections Full time📌 Hiring: Data Engineer | 9-12 Years Experience | Remote (India) 💰 Salary: Up to 26 LPA 🛠️ Skills: Python, SQL, Big Data, Cloud, SnowflakeJob Description: We are looking for an experienced Data Engineer with strong hands-on expertise in designing and building scalable data pipelines and cloud-based data solutions. The ideal candidate will have...