Data Pipeline Specialist

2 weeks ago

Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 25,00,000

Key Roles and Responsibilities

  • We are seeking a skilled professional to design and implement reliable, scalable, and efficient data pipelines using modern DevOps tools and technologies.
  • The ideal candidate will be responsible for designing, implementing, and managing Continuous Integration/Continuous Deployment (CI/CD) pipelines using GitHub Actions or equivalent tools.
  • Additionally, they will configure and maintain GitHub branch protection rules and workflows to ensure smooth code integration and deployment.
  • The successful candidate will also build, optimize, and debug large-scale data processing jobs in Databricks (Spark) using Python and SQL.
  • They will automate Databricks deployments using Databricks Asset Bundles (DABs) or similar frameworks.
  • Maintaining robust data pipelines is another crucial responsibility of the role.
  • This includes developing, testing, and maintaining data pipelines using Python and SQL, as well as implementing unit testing, validation, and quality checks for data workflows.
  • Another key aspect of the job is managing multi-environment deployments (Dev, Stage, Prod) ensuring reliability and consistency.
  • Apart from that, the person will write and maintain configuration files (YAML, JSON) for workflows, pipelines, and automation.
  • Troubleshooting and resolving configuration or syntax issues in YAML and related files is also part of the job description.
  • Moreover, the person will partner with cross-functional teams to understand needs and deliver data models, insights, and reporting they can trust.
Requirements
  • Familiarity with SQL, Databricks (Spark), and Python/R programming languages.
  • Understanding of GitHub and CI/CD practices (branch protection, GitHub Actions workflows).
  • Experience in implementing unit tests and quality checks in data workflows.
  • Knowledge of Databricks Asset Bundles (DABs) or similar automation tools.
  • Familiarity with YAML configuration management, debugging, and error resolution.
  • Strong problem-solving and debugging skills in data engineering workflows.
  • Hands-on experience with Snowflake or another cloud data warehouse.
  • Familiarity with Tableau/Power BI dashboards and a good sense for usability.

  • Data Engineering Specialist

    2 weeks ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 25,00,000

    Job Title: Data Engineering SpecialistWe are seeking a highly skilled professional to design, develop, and maintain large-scale data acquisition pipelines in the Google Cloud Ecosystem.Key Responsibilities:Develop and maintain data acquisition pipelines for structured and unstructured data using tools such as Python, Data Flow, DataStream, CDC (Change Data...

  • Expert Data Pipeline Developer

    2 weeks ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 21,90,000 - ₹ 25,19,000

    Unlock Your Potential as a Data ExpertThis is an exciting opportunity for a skilled professional to join our operational intelligence platform, which leverages AI/ML-based analytics to provide actionable insights to network, cloud, and application operators.We empower operations teams to meet their KPIs through seamless collaboration, search-driven...

  • Data Ingestion Specialist

    2 weeks ago

    Tirupati, Andhra Pradesh, India beBeeDataIngestion Full time ₹ 90,00,000 - ₹ 1,50,00,000

    Job Title: Data Ingestion SpecialistThis role requires a highly skilled professional with expertise in data ingestion processes, focusing on integrating various data sources into Databricks.The ideal candidate will have a strong understanding of data ingestion pipelines, CI/CD tools, and Kubernetes. They will design, develop, and optimize data ingestion...

  • Automated Data Simulation Specialist

    2 weeks ago

    Tirupati, Andhra Pradesh, India beBeeSimulation Full time ₹ 14,53,708 - ₹ 24,31,230

    Job Title: Automated Data Simulation SpecialistWe are seeking a detail-oriented and highly skilled professional to join our data analytics team.In this role, you will be responsible for the development of automated scripts in Python to create a golden model within SQL Server for use in test automation.You will work closely with data engineers, QA analysts,...

  • Azure Integration Specialist

    2 weeks ago

    Tirupati, Andhra Pradesh, India beBeeIntegration Full time ₹ 6,00,000 - ₹ 10,00,000

    Azure Integration SpecialistWe are seeking a highly skilled Azure Integration Specialist to design, develop and deploy data pipelines using Azure Data Factory (ADF) and other Azure services.Key responsibilities include designing, developing and deploying data pipelines, troubleshooting and performance optimization for Azure Synapse Analytics, Power BI and...

  • Cloud Computing Specialist

    2 weeks ago

    Tirupati, Andhra Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Cloud Computing SpecialistWe are seeking a skilled Cloud Computing Specialist to join our team. In this role, you will be responsible for developing and maintaining large-scale data pipelines using Azure Data Factory and other Azure Data Services.Key Responsibilities:Develop and maintain efficient data integration and transformation processes.Design and...

  • Data Engineering Specialist

    2 weeks ago

    Tirupati, Andhra Pradesh, India beBeeCloud Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OverviewLTIMindtree seeks an experienced and results-driven Tech Lead to lead the development and deployment of cloud-based data solutions. This role involves overseeing the technical direction of projects, mentoring team members, and delivering scalable and efficient systems using cloud and big data technologies.Key Responsibilities:Design, develop, and...

  • Chief Cloud Data Specialist

    2 weeks ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Cloud Data EngineerJob DescriptionAs a Cloud Data Engineer, you will play a pivotal role in designing and developing scalable ETL pipelines using Azure services. Your primary responsibility will be to process, transform, and load large datasets into AWS Datalake or other data stores.Key ResponsibilitiesDesign and develop cloud-based ETL pipelines using Azure...

  • Senior Automation Governance Specialist

    2 weeks ago

    Tirupati, Andhra Pradesh, India beBeeGovernance Full time ₹ 90,00,000 - ₹ 1,20,00,000

    Senior Automation Governance SpecialistThe organization seeks a seasoned professional to establish and implement an automation governance platform, ensuring adherence to best practices across the enterprise.Key ResponsibilitiesDevelop a framework for validating code and practices, enhancing efficiency and accuracy.Create a testing suite-like framework for...

  • Senior Cloud Data Architecture Specialist

    2 weeks ago

    Tirupati, Andhra Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Cloud Data Engineer PositionWe are seeking a highly skilled Cloud Data Engineer to join our team and take on the challenge of designing and implementing scalable data pipelines, data lakes, and data warehouses.The ideal candidate will have extensive experience with cloud-native ETL tools, AWS, Azure, and GCP, as well as strong proficiency in data platforms...