Senior Data Engineering Position

1 week ago

palakkad, India beBeeDataEngineer Full time

Data Engineering LeadWe are building the future of healthcare analytics. Our goal is to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems. These pipelines must be reliable, observable, and continuously improving in production. This role is fully remote, open to candidates based in Europe or India, with periodic team gatherings in Mountain View, California. Main Responsibilities: Design, build, and maintain scalable ETL pipelines using Python (Pandas, PySpark) and SQL, orchestrated with Airflow (MWAA). Develop and maintain the SAIVA Data Lake/Lakehouse on AWS, ensuring quality, governance, scalability, and accessibility. Run and optimize distributed data processing jobs with Spark on AWS EMR and/or EKS. Implement batch and streaming ingestion frameworks (APIs, databases, files, event streams). Enforce validation and quality checks to ensure reliable analytics and ML readiness. Monitor and troubleshoot pipelines with CloudWatch, integrating observability tools like Grafana, Prometheus, or Datadog. Automate infrastructure provisioning with Terraform, following AWS best practices. Manage SQL Server, PostgreSQL, and Snowflake integrations into the Lakehouse. Participate in an on-call rotation to support pipeline health and resolve incidents quickly. Write production-grade code, and contribute to design/code reviews and engineering best practices. Required Skills: 5+ years in data engineering, ETL pipeline development, or data platform roles (flexible for exceptional candidates). Experience designing and operating data lakes or Lakehouse architectures on AWS (S3, Glue, Lake Formation, Delta Lake, Iceberg). Strong SQL skills with PostgreSQL, SQL Server, and at least one AWS cloud warehouse (Snowflake or Redshift). Proficiency in Python (Pandas, PySpark); Scala or Java a plus. Hands-on with Spark on AWS EMR and/or EKS for distributed processing. Strong background in Airflow (MWAA) for workflow orchestration. Expertise with AWS services: S3, Glue, Lambda, Athena, Step Functions, ECS, CloudWatch. Proficiency with Terraform for IaC; familiarity with Docker, ECS, and CI/CD pipelines. Experience building monitoring, validation, and alerting into pipelines with CloudWatch, Grafana, Prometheus, or Datadog. Strong communication skills and ability to collaborate with data scientists, analysts, and product teams. A track record of delivering production-ready, scalable AWS pipelines, not just prototypes.

  • Senior Data Engineer

    3 weeks ago

    Palakkad, India K&K Talents Full time

    K&K Talents is an international recruiting agency that has been providing technical resources globally since 1993. This position is with one of our clients in India, who is actively hiring candidates to expand their teams. Role : Senior Data Engineer (Tableau + Real Estate Domain) Locations: Pune | Bangalore | Hyderabad | Gurgaon | Chennai | Noida | Mumbai...

  • Senior Data Engineering Position

    2 weeks ago

    palakkad, India beBeeDataEngineer Full time

    Business Intelligence Data Engineer OpportunityWe're seeking an experienced Data Engineer to design, setup and manage large-scale data warehousing projects from initial planning through to production deployment.Develop and implement efficient ETL workflows for seamless data ingestion and processing.Create optimized schema designs and indexing strategies for...

  • Senior Data Engineer

    3 weeks ago

    Palakkad, India DataOrbit AI Full time

    Company Description DataOrbit AI is a leading AI consulting firm dedicated to empowering small and medium investment managers to achieve AI readiness. By uniting strategy, data preparation, technology, and execution, we enable organizations to adopt AI solutions that unlock growth opportunities and operational efficiencies. With expertise in AI strategy,...

  • Senior Data Platform Engineer

    3 days ago

    palakkad, India beBeeData Full time

    Job Title: Senior Data Platform EngineerWe're seeking a seasoned Software Engineer to join our team as a Senior Data Platform Engineer. This role will focus on building and maintaining the foundational data-heavy services that power our organization.About the Role:Design, build, and operate our cloud-based data storage systems, including data lakes and...

  • Senior Data Engineer

    4 weeks ago

    Palakkad, India Mindlance Full time

    We’re looking for a strong, hands-on Sr Data Engineer who can independently drive client conversations, collaborate with SMEs, and deliver high-quality data solutions in a cloud-native environment.🔹 Job Title: Senior Data Engineer (Databricks + DBT) – 100% Remote 🔹 Engagement: Full-time🔹 Must-Have Skills:• 8–10 years with Oracle Retail suite...

  • Senior Data Engineer

    4 weeks ago

    Palakkad, India Mindlance Full time

    We’re looking for a strong, hands-on Sr Data Engineer who can independently drive client conversations, collaborate with SMEs, and deliver high-quality data solutions in a cloud-native environment.🔹 Job Title: Senior Data Engineer (Databricks + DBT) – 100% Remote 🔹 Engagement: Full-time🔹 Must-Have Skills:• 8–10 years with Oracle Retail suite...

  • Master Data Scientist Position

    2 weeks ago

    palakkad, India beBeeDataScience Full time

    Senior Data Scientist RoleWe are seeking a seasoned Data Science Expert to join our team. The ideal candidate will possess strong machine learning, data engineering, and cloud computing skills, particularly with AWS.Predictive Modeling & Machine Learning:Develop and deploy machine learning models for forecasting, optimization, and predictive analytics using...

  • Senior Data Scientist Position

    4 days ago

    palakkad, India beBeeData Full time

    Machine Learning Engineer RoleKey responsibilities:Develop and implement machine learning models and algorithms for production systems.Build scalable ML training pipelines.Analyze large datasets, including data validation.The ideal candidate should have experience in writing Python code.Experience with software engineering principles.Bachelor's or Master's...

  • Senior Data Architect

    4 days ago

    palakkad, India beBeeData Full time

    Job Title: Data Architect & EngineerThe RoleWe seek a seasoned Senior Data Engineer to architect, build and own the data pipelines that power our large language model development.As a senior individual contributor, you will be the team's expert on data ingestion, processing and quality for all AI training.Key Responsibilities Include:Designing, developing,...

  • Senior Master Data Engineer

    5 days ago

    palakkad, India beBeeDataEngineer Full time

    Job OverviewThis is a remote full-time position for an experienced data engineer. The successful candidate will be responsible for designing, implementing, and maintaining master data solutions with a focus on Tibco EBX technologies.Key ResponsibilitiesDeveloping and managing data pipelines to ensure seamless data flow.Designing and implementing data models...