Senior Data Pipeline Specialist

3 days ago

Aligarh, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

Are you a seasoned Data Engineer looking for a challenging project? We're seeking an experienced professional to work on building robust data pipelines on the Google Cloud Platform.

This is a remote opportunity for a specialist with 10+ years of experience to work on projects for industries like Energy, Oil & Gas, Healthcare, and Finance.

As a GCP Data Engineer, your key responsibilities will include:

  • Designing, developing, and maintaining scalable data pipelines for both batch and real-time processing using GCP tools.
  • Constructing large, complex datasets to meet diverse business needs.
  • Translating business requirements into clear technical specifications.
  • Leveraging GCP services, including Big Query, Cloud Dataflow (Apache Beam), Cloud Run, Cloud Functions, and Pub/Sub to build integrated systems.
  • Orchestrating workflows using Cloud Composer (Apache Airflow).
  • Implementing DevOps and CI/CD practices to ensure the reliability and scalability of data pipelines.
  • Ensuring data security and governance using IAM roles, policies, and data masking techniques.
  • Monitoring pipelines using logs and alerts to proactively manage performance.

The ideal candidate will have deep expertise in the following areas:

GCP Services:

  • Big Query
  • Cloud Dataflow (Apache Beam)
  • Cloud Run, Cloud Functions, Cloud Storage
  • Pub/Sub, Cloud Composer (Apache Airflow)
  • Experience with Change Data Capture (CDC) and Data stream is a plus.

Technical Skills:

  • Proficiency in Python and SQL.
  • Experience with structured ERP solutions like SAP, Oracle EBS, and Salesforce.
  • A strong understanding of different file formats (CSV, JSON, XML, Parquet, ORC, Avro).
  • Knowledge of Data Lake concepts.
  • Experience with API development and consumption (REST, SOAP).

Data Security & Governance:

  • Familiarity with data masking and anonymization techniques.

About this role

We empower companies to become truly data-driven by providing comprehensive solutions in Data Strategy, Governance, Advanced Analytics, and Data Visualization. Our tailored approach and commitment to excellence help our clients drive digital transformation and build a competitive edge.

To apply, please share your updated resume with us.

  • Data Pipeline Engineer

    3 days ago

    Aligarh, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Data Pipeline SpecialistSummary: An experienced data pipeline professional with 6+ years of expertise in designing and maintaining scalable data engineering projects.About the Role:Collaborate with stakeholders to comprehend data requirements and translate them into technical solutions.Design, develop, and deploy data pipelines and ETL processes using...

  • Senior Data Pipeline Specialist

    1 week ago

    Aligarh, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 24,00,000

    Data Engineer PositionAs a senior data engineer, you will play a critical role in designing and implementing data pipelines that drive business growth. Your expertise in Python and AWS will enable you to develop scalable and efficient solutions for large datasets.The ideal candidate will have strong knowledge of ETL tools such as Glue and Talend, as well as...

  • Data Pipeline Specialist

    4 days ago

    Aligarh, Uttar Pradesh, India beBeeData Full time ₹ 9,00,000 - ₹ 15,00,000

    System Tester Job DescriptionJob Title:ETL TesterAbout the Role:We are seeking a highly skilled QA Engineer with expertise in ETL testing, Kafka/Redpanda, and MongoDB.The role involves testing large-scale data pipelines, ensuring data accuracy, consistency, and throughput in event-driven systems.Key Responsibilities:Test ETL processes and perform data...

  • Data Ingestion Pipeline Specialist

    2 weeks ago

    Aligarh, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 8,00,000 - ₹ 15,00,000

    Highly skilled Data Engineers with a strong focus on data ingestion are sought after to develop scalable and efficient ingestion workflows.Data Ingestion EngineerWe are seeking an experienced professional to lead the design, development, and optimization of data pipelines for integrating multiple sources into Databricks.Design, implement, and maintain data...

  • Senior Data Pipeline Architect

    1 week ago

    Aligarh, Uttar Pradesh, India beBeeDataIntegrator Full time ₹ 60,00,000 - ₹ 1,00,00,000

    Key Data Integration RoleJob Summary:We are seeking a seasoned data integration professional to join our team. As an O9 Integration Consultant, you will be responsible for designing and building robust data pipelines to support business operations.The ideal candidate will have extensive experience in ETL processes, proficiency in databases, and strong...

  • Pipeline Operations Specialist

    6 days ago

    Aligarh, Uttar Pradesh, India beBeePipeline Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Pipeline Operations SpecialistThis role involves conducting hot tapping and pipeline isolation services, ensuring safe and efficient operations while adhering to industry standards and regulations.Perform site inspections and prepare technical reportsCoordinate with clients to meet their requirementsThe ideal candidate will have experience in hot...

  • Senior Data Pipeline Architect

    1 week ago

    Aligarh, Uttar Pradesh, India beBeeEngineer Full time ₹ 1,60,00,000 - ₹ 2,01,00,000

    Lead Python Platform EngineerYou will be responsible for designing and building scalable data pipelines using Apache Beam.Design and build a shared component library/SDK for ingestion, parsing/OCR, extraction, validation, enrichment, and publishing.Define patterns/templates for Apache Beam pipelines and Databricks jobs; standardize configuration, packaging,...

  • Senior Data Migration Specialist

    2 weeks ago

    Aligarh, Uttar Pradesh, India beBeeDataMigration Full time ₹ 15,00,000 - ₹ 20,00,000

    Senior Data Migration SpecialistWe are seeking an experienced data migration specialist to lead comprehensive data migration initiatives. The successful candidate will assess legacy systems, formulate effective migration strategies, and ensure seamless business continuity.

  • Data Pipeline Validator

    4 days ago

    Aligarh, Uttar Pradesh, India beBeeETL Tester Full time ₹ 9,00,000 - ₹ 12,00,000

    Job OpportunityWe are seeking a skilled professional to validate data pipelines and transformations in Databricks, utilizing Python scripting and strong SQL skills.About the Role:This role involves working with complex data sets, identifying discrepancies, and implementing corrective actions to ensure seamless data flow.Key Responsibilities:

  • Data Pipeline Architect

    3 days ago

    Aligarh, Uttar Pradesh, India beBeeDataEngineering Full time US$ 1,00,000 - US$ 1,50,000

    Data Engineer Role Overview">At the heart of every innovative organization lies a robust data infrastructure, empowering decision-makers to drive growth and optimize performance. As a Data Engineer, you will play a pivotal role in designing and optimizing our distributed data pipelines, infrastructure, and tools that power insights across our business.">Key...