Senior Data Pipeline Specialist

4 days ago

kurnool, India beBeeDataEngineer Full time

We are seeking a seasoned professional to lead our efforts in designing, building, and scaling robust data pipelines that power nationwide analytics and support our machine learning systems.This is a fully remote role open to candidates based in Europe or India with periodic team gatherings in Mountain View California.Our goal is to create pipelines that are reliable observable and continuously improving in production.The Data Lake/Lakehouse on AWS ensures quality governance scalability and accessibility.We are looking for someone who can design build and maintain scalable ETL pipelines using Python (Pandas PySpark) and SQL orchestrated with Airflow (MWAA).We need someone who can run and optimize distributed data processing jobs with Spark on AWS EMR and/or EKS.Implementing batch and streaming ingestion frameworks (APIs databases files event streams) is crucial.Enforcing validation and quality checks to ensure reliable analytics and ML readiness is essential.Monitoring and troubleshooting pipelines with CloudWatch integrating observability tools like Grafana Prometheus or Datadog is required.Automating infrastructure provisioning with Terraform following AWS best practices is necessary.Requirements:5+ years of experience in data engineering ETL pipeline development or data platform roles.Experience designing and operating data lakes or Lakehouse architectures on AWS is required.Strong SQL skills with PostgreSQL SQL Server and at least one AWS cloud warehouse are necessary.Proficiency in Python (Pandas PySpark) Scala or Java is a plus.Hands-on experience with Spark on AWS EMR and/or EKS for distributed processing is required.Strong background in Airflow (MWAA) for workflow orchestration is necessary.Benefits:This role offers a unique opportunity to work with cutting-edge technologies and contribute to the development of innovative data solutions.Others:Please note that this role requires a strong understanding of data engineering principles and experience working with large-scale data systems.

  • Senior Data Pipeline Specialist

    2 weeks ago

    kurnool, India beBeeDataEngineer Full time

    Data Engineer Role Description:Data Engineers play a pivotal role in designing and developing scalable Extract, Transform, Load (ETL) workflows to facilitate seamless data integration, transformation, and migration across various systems.Key Responsibilities:Create efficient ETL processes using Talend to extract valuable insights from complex data...

  • Data Engineering Specialist

    1 week ago

    kurnool, India beBeeDataEngineering Full time

    Job Title:Data Engineering SpecialistAbout the Role:We are seeking a highly skilled Data Engineering Specialist to join our team. As a key member of our data engineering team, you will be responsible for designing and maintaining scalable, efficient, and well-partitioned schemas in MSSQL, Redshift, and Snowflake.You will work closely with software engineers,...

  • Chief Data Infrastructure Specialist

    1 week ago

    kurnool, India beBeeDataEngineer Full time

    Senior Data Engineer PositionJob Description:We are seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, building, and optimizing high-performance data pipelines and data systems.Key Responsibilities:Design, develop, and maintain scalable and reliable ETL/ELT pipelines.Build data...

  • Data Integration Specialist

    6 days ago

    kurnool, India beBeeIntegration Full time

    Job Title: Data Integration SpecialistWe are seeking an experienced Data Integration Specialist to join our team. The ideal candidate will have a strong background in developing and maintaining ETL processes, reports, and database solutions.This role requires a high degree of technical expertise, excellent problem-solving skills, and the ability to...

  • Senior Data Architect

    2 weeks ago

    kurnool, India beBeeData Full time

    Unlock Your Potential as a Senior Data ArchitectWe are seeking an experienced professional to lead the design, development, and optimization of robust data pipelines across on-prem and cloud ecosystems. As a Senior Data Architect at our organization, you will be responsible for architecting efficient data ingestion workflows from varied sources including...

  • Chief Data Pipeline Architect

    2 weeks ago

    kurnool, India beBeeDataEngineer Full time

    Data Engineer Job DescriptionWe are seeking a skilled data engineer to join our team and help us deliver a multi-year project initiative using Palantir Foundry, PySpark, and TypeScript. This is a long-term contract for a full-time position, with the opportunity to work remotely and live anywhere with an internet connection.The successful candidate will be...

  • Senior Backend Data Specialist

    2 weeks ago

    kurnool, India beBeeDataEngineer Full time

    Freelance Data Engineer Role As a Freelance Data Engineer, you will work on data-intensive applications and perform hands-on development for data pipelines and database performance tasks. The ideal candidate will possess strong analytical skills and be proficient in writing efficient SQL queries to optimize data performance and troubleshoot issues....

  • Senior Data Engineering Specialist

    6 days ago

    kurnool, India beBeeDataEngineer Full time

    Key ResponsibilitiesData engineering involves designing and implementing end-to-end data pipelines following architecture patterns.Data transformations using SQL for large-scale data processing are key to our operations.Developing dimensional data models, aggregations, and KPI calculations in analytics consumption is crucial.We need someone to implement data...

  • Senior Data Engineer

    2 weeks ago

    Kurnool, India Mitra AI Full time

    We are seeking a skilled and proactive Senior Engineer with strong Snowflake expertise and AWS cloud exposure to lead the implementation and operational excellence of a scalable, multitenant modern data platform for a leading US-based marketing agency serving nonprofit clients. This role requires hands-on experience in managing Snowflake environments,...

  • Influencer Pipeline Specialist

    6 days ago

    kurnool, India beBeeInfluencer Full time

    Role OverviewAs an Influencer Coordinator, you will play a pivotal role in supporting influencers and pipeline development. This full-time position requires a minimum of 35 hours per week, with flexibility to adapt to changing business needs.This role involves identifying and partnering with influential individuals, managing databases, and executing general...