Senior Data Pipeline Specialist

2 days ago

Tiruppur, Tamil Nadu, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000
Job Title

Big Data Architect

Role Description:

  • As a senior data engineer, you will lead the design, development and deployment of scalable data pipelines using Apache Spark.
  • Developing robust data processing workflows in PySpark is essential for delivering insights to our stakeholders.
  • You will collaborate with cross-functional teams to ensure alignment on data requirements and deliver high-quality solutions.
  • Designing and implementing data ingestion, transformation and integration solutions is crucial for unlocking business value from our data assets.
  • Optimizing Spark jobs for performance and cost efficiency is key to delivering results efficiently.
  • Leading a team of engineers ensures best practices are adopted across the board.

Required Qualifications:

  • Bachelor's or Master's degree in Computer Science Engineering or a related field.
  • 10+ years of professional experience in data engineering.
  • Strong hands-on programming experience in Scala and Python is essential.
  • Extensive experience with Apache Spark, including Spark SQL, Spark Streaming, and optimization techniques.
  • Proficiency in PySpark for data processing tasks is required.
  • Experience working with cloud-based platforms such as Databricks.
  • Solid understanding of distributed computing, big data architectures, and data modeling is necessary.
  • Experience with cloud platforms such as AWS, Azure, or GCP.
  • Familiarity with data storage technologies like HDFS, S3, Delta Lake, and relational/non-relational databases.
  • Strong leadership skills with experience leading and mentoring teams.
  • Excellent problem-solving, communication, and collaboration skills.

  • Data Pipeline Architect

    1 day ago

    Tiruppur, Tamil Nadu, India beBeeData Full time ₹ 18,00,000 - ₹ 23,10,000

    Senior Data Engineer Job DescriptionWe are seeking a seasoned data engineer to design and optimize complex data pipelines. The ideal candidate will have a strong background in ETL, Cloud, Data Warehousing, and Big Data ecosystems.Key Responsibilities:Design and Optimize Data Pipelines: Build ETL/ELT workflows using AWS Glue, Airflow, Databricks, StreamSets,...

  • Data Pipeline Specialist

    1 day ago

    Tiruppur, Tamil Nadu, India beBeeData Full time US$ 1,00,000 - US$ 1,50,000

    Job Role:We are seeking a skilled professional to fill the position of Data Engineer.Responsibilities:Develop and maintain large-scale data pipelines using Python, PySpark, SQL, and Bash scripting.Process both structured and unstructured data.Collaborate with cross-functional teams to implement ETL processes and data transformations.Stay updated with...

  • Senior Data Pipeline Developer

    2 weeks ago

    Tiruppur, Tamil Nadu, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    We are seeking a skilled Data Engineer to join our team.The successful candidate will be responsible for designing and developing scalable data pipelines using IBM DataStage and AWS Glue.They will work closely with architects, business analysts, and data modelers to ensure timely and accurate delivery of critical data assets.Key ResponsibilitiesDesign,...

  • Senior Data Pipeline Validator

    2 days ago

    Tiruppur, Tamil Nadu, India beBeeDataQuality Full time ₹ 18,00,000 - ₹ 22,00,000

    ETL Data Quality EngineerWe are seeking a meticulous ETL Data Quality Engineer with expertise in testing large-scale data pipelines and ensuring data accuracy, consistency, and throughput in event-driven systems.Key Responsibilities:Test ETL processes and perform data reconciliation across systems to guarantee seamless data flow.Validate Kafka/Redpanda...

  • Data Pipeline Engineer

    1 week ago

    Tiruppur, Tamil Nadu, India beBeeDeveloper Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Talend ETL DeveloperRequired Skills and Qualifications:Design and optimize complex data pipelines using TMC.Proficiency in BigQuery, including partitioning, clustering, federated queries, and advanced SQL.Expertise in Python for ETL scripting, automation, and integration with cloud data pipelines.Hands-on experience with PostgreSQL, including...

  • Senior Data Integration Specialist

    1 day ago

    Tiruppur, Tamil Nadu, India beBeeDataSpecialist Full time ₹ 9,00,000 - ₹ 15,00,000

    Job OverviewThis is a full-time role for an ETL specialist with a renowned organization.The ideal candidate will be responsible for designing, deploying, and managing data integration processes using various tools. Key tasks include utilizing ETL software, performing data modeling, ensuring data accuracy, and providing analytical insights. Additionally, the...

  • Senior Pipeline Developer

    4 days ago

    Tiruppur, Tamil Nadu, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    As a Data Engineer, you will play a pivotal role in building scalable pipelines and working with streaming data to deliver high-quality solutions.About the RoleThis position involves designing, building, and optimizing data pipelines and ETL workflows.You will manage and process large datasets using Spark, PySpark, and SQL.Experience with real-time streaming...

  • Senior Data Integration Specialist

    12 hours ago

    Tiruppur, Tamil Nadu, India beBeeAbinitiodeveloper Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job OverviewWe are seeking a skilled professional to fill the position of Senior Data Integration Specialist.The ideal candidate will possess extensive experience in designing and implementing data integration solutions using Ab Initio, with a strong focus on delivering high-quality projects on time.About the Role:This is an exceptional opportunity for a...

  • Data Transformation Specialist

    13 hours ago

    Tiruppur, Tamil Nadu, India beBeeDataTransformation Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Data Transformation SpecialistWe are seeking an experienced data transformation specialist to support our infrastructure, develop and maintain ETL pipelines for pricing, shipping, and behavioral datasets, and deploy dynamic pricing algorithms.

  • Chief Data Pipeline Architect

    5 days ago

    Tiruppur, Tamil Nadu, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 25,00,000

    Data Engineering OpportunityWe are looking for a skilled professional to lead our data engineering efforts. As a data engineer, you will be responsible for designing and implementing scalable data pipelines using modern tools and technologies.About the Role:Develop and maintain large-scale data processing systems using Spark and Databricks.Implement CI/CD...