Senior Data Engineering Lead

2 weeks ago

Belgaum, Karnataka, India beBeeDataEngineeringLead Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

Job Title: Senior Data Engineering Lead

We are seeking a highly skilled and experienced leader to join our team as Senior Data Engineering Lead. The ideal candidate will have a minimum of 4 years of hands-on experience in data engineering, with at least 2 years leading a team of 5+ developers.

Key Responsibilities:

  • Design and develop scalable data solutions using tools like Scrapy and Python libraries.
  • Lead and mentor a team of 5+ developers, managing project timelines and deliverables.
  • Implement advanced blocking and captcha-solving techniques to bypass restrictions.
  • Conduct source code reverse engineering and automate web and app interactions.
  • Manage proxies, IP rotation, and SSL unpinning to ensure effective scraping.
  • Maintain and improve API integrations and data pipelines.
  • Ensure code quality through effective version control, error handling, and documentation.
  • Collaborate with cross-functional teams for project planning and execution.
  • Monitor performance and provide solutions under high-pressure environments.

Required Skills and Experience:

  • Data Engineering: Minimum 4 years in data engineering industry.
  • Leadership: Minimum 2 years leading a team of 5+ developers.
  • Scraping Tools: Scrapy, Threading, requests, web automation.
  • Technical Proficiency: Advanced Python, Captcha solving and blocking handling, Source reverse engineering, Proxy management & IP rotation, App automation, SSL Unpin, Frida, API Management, Version Control Systems, Error Handling, SQL, MongoDB, Pandas.
  • Leadership Skills: Basic project management, Moderate documentation, Team handling, Pressure management, Flexibility and adaptability, High accountability.

Preferred (Good to Have):

  • Experience with Linux.
  • Knowledge of Appium, Fiddler, Burp Suite.

Benefits:

  • Growth opportunities for senior leaders.

Others:

  • Positive work environment.

  • Senior Data Engineer

    2 weeks ago

    Belgaum, Karnataka, India beBeeAirflowExpert Full time ₹ 15,00,000 - ₹ 19,00,000

    Senior Data Engineer - Airflow ExpertWe are seeking a highly skilled Senior Data Engineer to lead the design and implementation of robust data pipelines using Airflow.The ideal candidate will have in-depth knowledge of Python, with experience in data engineering and cloud-based data warehousing solutions.Key responsibilities include:Designing and developing...

  • Data Engineering Lead

    2 weeks ago

    Belgaum, Karnataka, India beBeeEngineering Full time ₹ 30,00,000 - ₹ 40,00,000

    About our company :We are a leading, global technology firm that focuses on creating the most transparent and efficient path for advertiser budgets to become publisher revenue.Our proprietary contextual technology is at the forefront of enhancing Programmatic buying, the latest industry standard in ad buying for digital platforms.The platform powers major...

  • Lead Data Engineer

    2 weeks ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 20,00,000

    Looking for a Data Engineering Challenge? We're seeking an experienced professional to lead our data engineering efforts. Your exceptional technical skills will enable you to design, develop and maintain large-scale data systems that drive business growth.Key Responsibilities:Data Pipeline Design: Implement data pipelines using Azure/AWS Databricks to...

  • Senior Data Engineering Testing Lead

    2 weeks ago

    Belgaum, Karnataka, India beBeeDataQuality Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job DescriptionWe are seeking a seasoned professional to lead comprehensive testing strategies for complex data engineering pipelines and product delivery initiatives.This individual will drive quality assurance across integrated systems, data workflows, and customer-facing applications while coordinating cross-functional testing efforts.The successful...

  • Leading Financial Data Architect

    2 weeks ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Data Engineer - Financial SystemsTalent500 is looking for a seasoned Senior Data Engineer to shape next-generation financial data architecture using Snowflake, dbt, and modern data engineering practices.This role will focus on architecting and leading the implementation of enterprise-grade data warehouse solutions in Snowflake, ensuring scalability,...

  • Senior Cloud Data Architect

    2 weeks ago

    Belgaum, Karnataka, India beBeeDataEngineering Full time ₹ 30,00,000 - ₹ 40,00,000

    Lead Data Engineering ExpertWe are seeking a highly experienced and skilled Senior Data Engineering Lead with expertise in Databricks and cloud-based data engineering to lead our data engineering team.This leadership role requires a visionary who can design, develop, and manage scalable data infrastructure and pipelines while mentoring and inspiring a team...

  • Data Engineering Lead

    2 weeks ago

    Belgaum, Karnataka, India beBeeProject Full time ₹ 16,20,000 - ₹ 21,30,000

    About the Opportunity: We are seeking an accomplished Technical Project Manager to oversee and execute projects in data engineering and analytics.A key aspect of this role is managing cross-functional teams to deliver data platform, pipeline, and analytics initiatives that align with business objectives and meet project timelines.Key Responsibilities:Plan,...

  • Software Engineering Lead

    2 weeks ago

    Belgaum, Karnataka, India beBeeTechnical Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Technical ExpertWe are seeking a highly skilled Senior Technical Expert to join our team. This individual will be responsible for leading technical initiatives and providing expertise in software development, architecture, and engineering.Job Description:The Senior Technical Expert will lead the development of complex software systems, working closely...

  • Senior Data Engineering Professional

    2 weeks ago

    Belgaum, Karnataka, India beBeeEngineering Full time ₹ 17,56,000 - ₹ 21,56,000

    Data Engineering LeadAre you a seasoned data engineer with strong leadership skills? We're looking for a talented professional to lead our cross-functional teams in delivering high-quality software products using Agile methodologies.About the Role:Leverage your SQL expertise to drive data-driven decision-making and optimize processesCollaborate with...

  • Senior Technical Lead

    1 week ago

    Belgaum, Karnataka, India beBeeTechnical Full time ₹ 20,93,700 - ₹ 23,62,000

    Job Title: Senior Technical LeadWe are expanding our core engineering team and need a hands-on technical leader who can architect, deliver, and own business-critical platforms end-to-end.Key responsibilities include owning architecture & delivery of high-scale systems handling tens of thousands of requests per second. Additionally, you will design, build,...