Scalable Data Engineering Leader

2 weeks ago

ahmedabad, India beBeeDataEngineering Full time

Job Title: Scalable Data Engineering LeaderWe are seeking an experienced and strategic data engineering leader to lead our web crawling initiatives. As a seasoned professional in designing scalable data pipelines and optimizing infrastructure, you will be responsible for architecting, implementing, and optimizing robust crawling systems.Lead cross-functional teams of data engineers and web crawlers to ensure high-quality delivery and adherence to best practices.Design and implement scalable data pipelines using workflow orchestration tools like Airflow or Celery.Develop and maintain robust crawling systems that meet business requirements.Establish pipeline orchestration and define data quality measures to ensure accuracy and reliability.Collaborate with product, engineering, and analytics teams to translate data requirements into technical solutions.About the Role:This is an exciting opportunity for a motivated and results-driven individual to lead our data engineering efforts. You will work closely with our product and engineering teams to design and implement scalable data pipelines and optimize our web crawling infrastructure.Analyze and resolve complex technical issues related to data engineering and web crawling.Collaborate with team members to prioritize tasks and achieve project goals.Communicate technical ideas and solutions effectively to both technical and non-technical stakeholders.You will have the opportunity to work on a wide range of projects and technologies, from big data processing to cloud-based infrastructure.Requirements:To succeed in this role, you should have:A bachelor's or master's degree in computer science, engineering, or a related field.7-12 years of experience in data engineering, pipeline design, or large-scale web crawling systems.Demonstrated expertise in designing scalable data pipelines and optimizing infrastructure.Experience leading cross-functional teams and collaborating with product and engineering teams.Strong analytical and problem-solving skills, with the ability to communicate complex technical ideas effectively.Please note that we are looking for a candidate with significant industry experience and expertise in data engineering and web crawling.

  • Scalable Data Solutions Architect

    1 week ago

    ahmedabad, India beBeeDataEngineer Full time

    Job Title:Data EngineerWe are seeking a skilled Data Engineer to design and implement scalable data pipelines using Apache NiFi. The ideal candidate will leverage GCP services such as BigQuery, Pub/Sub, Dataflow, Dataproc, Cloud Storage, and Composer to develop real-time data ingestion pipelines for structured and unstructured datasets.

  • Big Data Engineering Leader

    7 days ago

    ahmedabad, India beBeeDataEngineer Full time

    Senior Data EngineerJoin a dynamic team in shaping the future of healthcare analytics. As a Senior Data Engineer, you will design and develop scalable data pipelines that power nationwide analytics and support machine learning systems.This is a fully remote role open to candidates based in Europe or India with periodic team gatherings in Mountain View,...

  • Group Leader

    4 weeks ago

    Ahmedabad, India ConsultMeIn Private Limited Full time

    Description :Role : Group Lead - Data EngineeringExperience : 15 - 18 YearsCTC : up to 90 LPA (including variable)Location : Ahmedabad (Work from office, Hybrid)Are you a data visionary ready to drive transformation at scale?My client is a GCC and is looking for an exceptional Group Lead Data Engineering to join their Global Capability Center (GCC) in India....

  • Software Quality Engineering Leader

    7 days ago

    ahmedabad, India beBeeQuality Full time

    At Millipixels Interactive, we're seeking a seasoned Quality Assurance Architect to drive our quality strategy and testing architecture. As part of our engineering ecosystem, you will bring deep expertise in modern testing frameworks, non-functional engineering, and quality-driven design.About the RoleThis role offers the opportunity to shape...

  • Building Scalable Data Infrastructure

    7 days ago

    ahmedabad, India beBeeData Full time

    Cloud Data ExpertLemongrass is a software-enabled services provider, synonymous with SAP on Cloud, focused on delivering superior, highly automated Managed Services to Enterprise customers. We partner with AWS, SAP, Microsoft and other global technology leaders.The ideal candidate will have extensive experience with cloud-native ETL tools such as AWS DMS,...

  • Senior Engineering Leader

    2 weeks ago

    ahmedabad, India beBeeEngineer Full time

    Engineering Leadership OpportunityWe're seeking a seasoned engineering leader to drive the development of our platform, shaping the future of hotel distribution at scale.Our tech stack includes Java (Spring Boot), Microservices, AWS ECS, ScyllaDB, MongoDB, Kafka, Redis, and more. As a hands-on leader, you'll be responsible for architecture and systems,...

  • Data Engineering Leader

    1 week ago

    ahmedabad, India beBeeDataEngineering Full time

    Data Engineering OpportunityWe are seeking an accomplished Data Engineer with a proven track record in large-scale data processing, cloud-based data platforms, and cutting-edge technologies to join our dynamic team.

  • AI Technology Leader

    2 weeks ago

    ahmedabad, India beBeeMachineLearning Full time

    Senior AI EngineerAbout the Position:This is a high-impact role, involving strategic responsibility for shaping the future of machine learning systems. As a senior AI engineer, you will be a hands-on technical leader who helps determine the company's machine learning infrastructure and system architecture.This position entails designing scalable AI...

  • Senior Data Engineer

    6 days ago

    Ahmedabad, Gujarat, India Maruti Techlabs Full time

    We are looking for aLead Data Engineerto drive the development of modern data platform. This role will focus on building scalable and reliable data pipelines using tools likeDBT,Snowflake, andApache Airflow, and will play a key part in shaping data architecture and strategy.As a technical leader, you'll work closely with cross-functional teams including...

  • Data Architecture Leader

    2 weeks ago

    ahmedabad, India beBeeDataArchitecture Full time

    About the JobWe are seeking a highly skilled Data Architect to design and develop robust data acquisition frameworks that support large-scale analytics and operational data systems.You will be responsible for building scalable ETL/ELT pipelines using Databricks (PySpark/Scala) and integrate with Snowflake for analytics workloads.Job TitleData Architect –...