Expertise in Large-Scale Data Engineering

3 days ago

Kollam, Kerala, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 14,00,000

Job Title: Data Engineer

We are seeking an experienced Data Engineer with strong expertise in CI/CD practices, Databricks (Spark), Python, and SQL. The ideal candidate should have hands-on experience in building and automating data pipelines, managing multi-environment deployments, and working with modern DevOps/configuration management tools.

Main Responsibilities:
  • Design, implement, and manage Continuous Integration/Continuous Deployment (CI/CD) pipelines using GitHub Actions or equivalent tools.
  • Configure and maintain GitHub branch protection rules and workflows to ensure smooth code integration and deployment.
  • Build, optimize, and debug Spark jobs in Databricks for large-scale data processing.
  • Automate Databricks deployments using Databricks Asset Bundles (DABs) or similar frameworks.
  • Develop, test, and maintain robust data pipelines using Python and SQL.
  • Implement unit testing, validation, and quality checks for data workflows.
  • Manage multi-environment deployments (Dev, Stage, Prod) ensuring reliability and consistency.
  • Write and maintain configuration files (YAML, JSON) for workflows, pipelines, and automation.
  • Troubleshoot and resolve configuration or syntax issues in YAML and related files.
  • Keep critical Databricks notebooks running reliably to ensure business continuity.
  • Migrate legacy reports into modern, self-serve dashboards (Snowflake, Tableau, Power BI).
Required Skills and Qualifications:
  • Strong skills in SQL, Databricks (Spark), and Python/R.
  • Strong knowledge of GitHub and CI/CD practices (branch protection, GitHub Actions workflows).
  • Experience in implementing unit tests and quality checks in data workflows.
  • Knowledge of Databricks Asset Bundles (DABs) or similar automation tools.
  • Familiarity with YAML configuration management, debugging, and error resolution.
  • Strong problem-solving and debugging skills in data engineering workflows.
  • Experience with Snowflake or another cloud data warehouse.
  • Hands-on experience with Tableau/Power BI dashboards and a good sense for usability.

  • Senior Data Engineering Strategist

    3 days ago

    Kollam, Kerala, India beBeeDataEngineering Full time ₹ 19,49,900 - ₹ 24,99,900

    Lead Data Engineer-Solution Architect RoleThis is a leadership position for an experienced Data Engineer who will be responsible for designing and implementing large-scale data architecture solutions.The ideal candidate will have a strong background in data architecture, software engineering, and solution delivery.A Bachelor's degree in Computer Science or...

  • Leader in Data Engineering

    2 days ago

    Kollam, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 40,00,000

    Senior Data Engineer PositionAt Tredence, we're looking for a skilled Senior Data Engineer to fill a key role in our data engineering team. The successful candidate will be responsible for designing and implementing large-scale data systems that enable clients to make informed business decisions.Responsibilities:Develop modern data warehouse solutions using...

  • Data Engineering Cloud Specialist

    2 days ago

    Kollam, Kerala, India beBeeCloudEngineer Full time ₹ 80,00,000 - ₹ 2,00,00,000

    Cloud Data Engineer Role Overview">We are looking for a highly skilled professional to join our team as a Cloud Data Engineer. This is an excellent opportunity to utilize your expertise in cloud-based technologies, specifically Google Cloud Platform (GCP), and work on designing, developing, and maintaining large-scale data processing systems.Key...

  • Highly Skilled Data Solutions Architect

    7 days ago

    Kollam, Kerala, India beBeeDataEngineering Full time ₹ 1,20,00,000 - ₹ 1,80,00,000

    Job DescriptionWe are seeking a seasoned Data Engineer to join our team. As a key member of our data engineering group, you will be responsible for designing and developing large-scale data processing systems using Python, PySpark, and Palantir Foundry.This is an exciting opportunity to bring your expertise in data engineering to drive business outcomes. You...

  • Data engineer

    3 days ago

    Kollam, Kerala, India Centrilogic Full time

    Data EngineerPurpose:Over 15 years, we have become a premier global provider of multi-cloud management, cloud-native application development solutions, and strategic end-to-end digital transformation services.Headquartered in Canada and with regional headquarters in the U. S. and the United Kingdom, Centrilogic delivers smart, streamlined solutions to...

  • Senior Data Engineering Professional

    5 days ago

    Kollam, Kerala, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 20,10,000

    Job Title: Senior Data EngineerWe are seeking an experienced professional to join our organization in the role of Senior Data Engineer. In this position, you will be responsible for developing and implementing data engineering solutions using Python and Pyspark.Key Responsibilities:Design and implement scalable data architectures to support large-scale data...

  • Data Architect

    10 hours ago

    Kollam, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Unlock Business Growth with Data Architecture Expertise">Data Strategy & Governance LeadershipLead the design and implementation of modern data platforms, including cloud-based solutions like Snowflake and Databricks, to drive business outcomes.Enable AI/ML capabilities for personalized customer experiences, forecasting, and analytics across the...

  • Data Solution Specialist

    3 days ago

    Kollam, Kerala, India beBeeArtificial Full time ₹ 30,00,000 - ₹ 40,00,000

    Expertise in AI and Machine LearningWe are looking for a seasoned professional to spearhead the development of AI-driven solutions using the Azure ecosystem.Key Responsibilities:Design, build, and deploy end-to-end ML pipelines encompassing data preparation, model training, validation, and deployment.Apply Large Language Models and Generative AI techniques...

  • Advanced Big Data Solutions Expert

    2 days ago

    Kollam, Kerala, India beBeeDataEngineering Full time US$ 75,000 - US$ 1,25,000

    Freelance Data EngineerWe are seeking an experienced Freelance Data Engineer to design, build, and optimize large-scale data pipelines. The consultant will collaborate with internal teams and clients to ensure robust, scalable, and high-performance data solutions.Design, develop, and maintain ETL/ELT pipelines using PySpark, AWS, and Databricks.Perform data...

  • Data Scientist/ML Engineer

    6 days ago

    Kollam, Kerala, India Quant-data Full time

    We're Hiring: Machine Learning Engineer / Data Engineer (Remote | Full-Time) Build AI-powered credit decisioning systems on Microsoft AzureWe're looking for a Machine Learning Engineer / Data Engineer with 5+ years of experience to join our AI-driven credit lending platform team. In this role, you'll design and deploy scalable ML solutions that power loan...