Data Pipeline Specialist

5 hours ago

Erode, Tamil Nadu, India beBeeDataPipelineSpecialist Full time ₹ 12,00,000 - ₹ 21,50,000

Job Title: Data Pipeline Specialist

Are you a skilled engineer with expertise in data pipeline architecture and big data technologies? Do you have hands-on experience with Spark, DBT, and cloud services?

We are looking for a talented individual to join our team as a Data Pipeline Specialist.

  • Create and maintain optimal data pipeline architectures
  • Design and implement event-driven architecture
  • Build infrastructure for efficient data extraction, transformation, and loading
  • Develop tools for analytics and data science teams
  • Collaborate with data scientists to improve data systems
Requirements
  • Proficiency in Scala or Python programming languages
  • Expertise in Apache Spark and performance tuning
  • Hands-on experience with DBT for data transformations and modeling
  • Experience with big data tools and relational databases
  • Knowledge of cloud services and stream-processing systems
Preferred Qualifications
  • Experience with HDFS, S3, Spark-Streaming, Flink, Hive, HBase, Kafka, Kinesis, Elasticsearch, Cassandra, MongoDB, Azkaban, Luigi, Airflow
  • Strong analytic skills for working with structured and unstructured datasets
  • Ability to handle large datasets efficiently and optimize Spark jobs
  • Experience contributing to open-source projects and having weekend side projects on GitHub
Why Join Us

At our company, we offer a dynamic environment where you can work with cross-functional teams, learn new technologies, and grow your career. We value innovation, collaboration, and continuous learning.

Join us and become part of a team that is passionate about delivering high-quality solutions and making a meaningful impact.

  • Data Pipeline Specialist

    4 hours ago

    Erode, Tamil Nadu, India beBeeDataEngineering Full time ₹ 10,00,000 - ₹ 20,00,000

    Job Description:Seeking a data engineering professional to build scalable data pipelines and cloud-native data solutions on GCP. Ideal candidate will be proficient in Pub/Sub, Dataflow, Cloud Storage, and BigQuery, with a foundational understanding of AI/ML workflows using Vertex AI.Key Responsibilities:Design and develop robust data ingestion pipelines...

  • Data Pipeline Specialist

    16 hours ago

    Erode, Tamil Nadu, India beBeeIntegration Full time US$ 90,000 - US$ 1,20,000

    Job Title:Azure Integration EngineerKey Responsibilities:Troubleshoot and resolve issues with Azure Synapse Analytics, Power BI, SQL Server, and Azure Data Factory (ADF)Optimize performance and tune queries for data pipelines and integrationsDesign, develop, and deploy data pipelines using ADFMaintain existing reports and databasesWork & Engagement...

  • Data Pipeline Specialist

    3 days ago

    Erode, Tamil Nadu, India beBeeDataQuality Full time ₹ 12,00,000 - ₹ 17,00,000

    About People Prime Worldwide:We are a global IT services company that specializes in digital engineering and IT services, helping clients modernize their technology infrastructure and accelerate innovation.Our expertise lies in combining deep industry knowledge with agile development practices to deliver scalable and cost-effective digital transformation...

  • Data Pipeline Specialist

    3 days ago

    Erode, Tamil Nadu, India beBeeelasticsearch Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    About this Position:We are looking for a skilled IT professional to maintain the health of our data pipeline.

  • Data Pipeline Quality Assurance Specialist

    7 days ago

    Erode, Tamil Nadu, India beBeeSoftwareTester Full time ₹ 25,00,000 - ₹ 30,00,000

    Job Title: Software Quality Assurance SpecialistJob Description:We are seeking a skilled Software Tester to join our team. As a QA/ETL Tester, you will be responsible for ensuring the quality and integrity of our data pipelines.The ideal candidate will have a strong background in data migration testing, with experience in source-to-target validation, record...

  • Senior Data Pipeline Specialist

    1 week ago

    Erode, Tamil Nadu, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineering OpportunityWe are seeking a skilled and detail-oriented data engineer to join our team and contribute to the design, build, and management of scalable data pipelines and enterprise data solutions.

  • Chief Data Pipeline Architect

    3 days ago

    Erode, Tamil Nadu, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    About the RoleWe are seeking a Senior Data Engineer to design, build, and operate scalable data pipelines.This is a critical role where you will create scalable pipelines that process clinical encounter data, retrieve associated medical documents, and deliver validated information to machine learning systems for inference.Your ResponsibilitiesDesign and...

  • Chief Data Pipeline Architect

    7 days ago

    Erode, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 30,00,000

    Key Roles and ResponsibilitiesWe are seeking a highly skilled professional to join our data engineering team. As a key member of the team, you will play a vital role in designing and implementing efficient data pipelines using modern DevOps tools.Develop and manage continuous integration and delivery pipelines using GitHub Actions or equivalent...

  • Expert Large Scale Data Pipeline Specialist

    7 days ago

    Erode, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Freelance Data Engineer – Python, SQL, AWSAbout the RoleWe are seeking an experienced Freelance Data Engineer with expertise in designing, building, and optimizing large-scale data pipelines using PySpark, AWS, and Databricks. The consultant will collaborate with internal teams and clients to ensure robust, scalable, and high-performance data...

  • Chief Data Pipeline Architect

    16 hours ago

    Erode, Tamil Nadu, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: Senior Data EngineerWe are seeking an exceptional Azure Data Engineer with 7+ years of experience to design, develop, and optimize data pipelines and integration solutions in a cloud environment.Key Responsibilities:Design and implement advanced ETL pipelines for batch data ingestion and egress.Develop scalable data solutions using Azure Data...