Data Pipeline Specialist

1 day ago

Tirupati, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

We are seeking a skilled data engineer to join our team as a Data Pipeline Specialist. The successful candidate will be responsible for designing, implementing and optimizing data pipelines using Azure Synapse, Databricks and Apache Spark (PySpark).

About the Role

The Data Pipeline Specialist will work closely with data architects, analysts and product owners to understand data requirements and deliver solutions. You will design and develop scalable data pipelines, integrate data from various sources, optimize data processing workflows for performance and cost-efficiency.

Key Responsibilities
  • Design, develop and maintain data pipelines using Azure Synapse, Databricks and Apache Spark (PySpark).
  • Integrate data from multiple sources ensuring data quality and consistency.
  • Optimize data processing workflows for performance and cost-efficiency.
  • Collaborate with cross-functional teams to understand data requirements and deliver solutions.
Requirements
  • Proficiency in Azure Synapse, Databricks and Apache Spark (PySpark).
  • Strong PySpark and SQL skills for data manipulation and querying.
  • Familiarity with Delta Live Tables and Databricks workflows.
  • Experience with ETL tools and processes.
  • Knowledge of cloud platforms (AWS, Azure, GCP).
  • Minimum 2 years of experience with Azure Synapse.
  • 5 years of experience in data engineering.
Skills and Qualifications
  • Excellent problem-solving abilities.
  • Strong communication and collaboration skills.
  • Ability to work in a fast-paced environment and manage multiple priorities.

  • Data Pipeline Specialist

    1 week ago

    Tirupati, Andhra Pradesh, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job OverviewAs a seasoned data pipeline specialist, we seek a highly skilled and motivated ETL developer to join our team.The ideal candidate will have extensive experience in designing, developing, and maintaining efficient data pipelines using IBM DataStage (Cloud Pak for Data) and AWS Glue/Lambda for ingestion from varied sources such as flat files, APIs,...

  • Freelance Data Pipeline Architect

    6 days ago

    Tirupati, Andhra Pradesh, India beBeeAzureDataFactory Full time ₹ 30,00,000 - ₹ 40,00,000

    We are seeking a highly skilled Freelance Azure Data Factory Specialist to support our data integration and ETL needs.About the RoleThe ideal candidate will have strong hands-on expertise in designing, building, and maintaining scalable and secure data pipelines across multiple sources and destinations using Azure Data Factory (ADF).Key...

  • Data Pipeline Specialist

    1 week ago

    Tirupati, Andhra Pradesh, India beBeeGenaideveloper Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Job OpportunityThe role of a Data Scientist involves leveraging data pipelining tools, reporting tools including PowerBI and Microsoft Excel, as well as Confluence.Key Responsibilities:Develop and maintain complex data pipelines using Python programming language.Drive business outcomes through GenAI concepts and use cases.Collaborate with cross-functional...

  • Data Pipeline Specialist

    1 day ago

    Tirupati, Andhra Pradesh, India beBeePipeline Full time ₹ 20,00,000 - ₹ 25,00,000

    We are seeking a skilled data professional to design and develop robust, scalable, and high-quality data pipelines that support analytics and reporting needs.Main Responsibilities:Develop and implement efficient data integration solutions that integrate structured and unstructured data from various sources into data lakes and warehouses.Collaborate with...

  • Chief Data Pipeline Architect

    1 week ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Title:Azure Data EngineerJob Description:We are seeking a skilled data engineer to develop, optimize and maintain automated data pipelines supporting governance and analytics initiatives.This role involves designing, building and deploying ETL/ELT pipelines in Databricks (PySpark, SQL, Delta Lake) to ingest, transform and curate metadata from multiple...

  • Data Pipeline Engineer

    3 days ago

    Tirupati, Andhra Pradesh, India beBeeElasticsearch Full time ₹ 10,40,000 - ₹ 15,40,000

    About this role:You will be responsible for ensuring the optimal performance and security of Elasticsearch clusters. This involves proactively identifying potential issues, planning and implementing upgrades, and extending monitoring and alert infrastructure.Expertise in administration and management of Elasticsearch clusters is required.Strong knowledge of...

  • Data Migration Specialist

    3 days ago

    Tirupati, Andhra Pradesh, India beBeeDataMigration Full time ₹ 45,00,000 - ₹ 90,00,000

    Data Migration SpecialistThe role of a Data Migration Specialist is pivotal in ensuring the seamless transition from legacy applications to modernized platforms. This position requires testing and validation of ETL pipelines and data warehouse migrations to guarantee data integrity and accuracy.

  • Senior Data Pipeline Architect

    3 days ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    We are seeking an experienced Data Engineer to design, develop and maintain data pipelines and infrastructure. As a key member of our team, you will work closely with data scientists, analysts and other stakeholders to ensure data is accurate, timely and accessible.Key Responsibilities:Design, develop and implement data pipelines using Azure Data Factory and...

  • Optimize Cloud Data Pipelines

    6 days ago

    Tirupati, Andhra Pradesh, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Freelance Data EngineerWe are seeking an experienced Freelance Data Engineer with expertise in Python and AWS services. The ideal candidate will design, develop, and optimize data pipelines using AWS Glue, PySpark, and Step Functions.

  • Chief Data Pipeline Architect

    14 hours ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    We're seeking a highly skilled Data Engineer to join our team and drive the design and optimization of our distributed data pipelines, infrastructure, and tools.This is an exciting opportunity for someone with 2-7 years of hands-on experience in data engineering to work on high-impact projects shaping the future of the industry.The ideal candidate will have...