Senior Python Data Pipeline Specialist

7 days ago

Bareilly, Uttar Pradesh, India beBeeEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000
Job Title:

Freelance Data Engineer – Python



\
Data Engineering Expertise Required

ThreatXIntel is a cyber security company that protects businesses and organizations from cyber threats.

We offer customized and affordable solutions tailored to meet the specific needs of our clients, ensuring that businesses of all sizes have access to high-quality cyber security services.

We are seeking an experienced Freelance Data Engineer with strong expertise in Python and AWS services (S3, Glue, Step Functions, PySpark) to design, develop, and optimize data pipelines.

  • Design and develop data pipelines using AWS Glue, PySpark, and Step Functions.
  • Implement ETL/ELT processes for ingesting, transforming, and delivering data from multiple sources.
  • Manage and optimize storage solutions with Amazon S3.
  • Write reusable and scalable Python code for data engineering workflows.
  • Implement unit and integration tests using pytest to ensure data quality and pipeline reliability.
  • Collaborate using Git/GitHub for version control, code reviews, and CI/CD integration.
  • Create clear technical documentation for pipelines, workflows, and design decisions.
  • Troubleshoot and resolve issues related to data pipelines and AWS services.

The ideal candidate will have 5–10 years of data engineering experience, with at least 5 years in the AWS ecosystem.

They will have strong coding skills in Python with experience in PySpark, hands-on experience with AWS Glue, Step Functions, and S3, proficiency in pytest for test automation, solid understanding of Git/GitHub workflows, and strong communication skills with a focus on documentation and knowledge sharing.

  • Senior Data Pipeline Specialist

    2 weeks ago

    Bareilly, Uttar Pradesh, India beBeeDataEngineer Full time US$ 40,800 - US$ 70,200

    Data Engineering Position We are seeking an experienced data engineering professional to join our team and contribute to the development of large-scale data pipelines using Python, PySpark, and TypeScript. This is a long-term contract opportunity that offers the chance to work remotely and be part of a dynamic team. Key Responsibilities: Develop and...

  • Lead Data Pipeline Specialist

    1 week ago

    Bareilly, Uttar Pradesh, India beBeeTester Full time ₹ 15,00,000 - ₹ 25,00,000

    About this position :We are seeking a highly skilled ETL and Power BI Tester to join our team. As an experienced professional, you will be responsible for ensuring the accuracy, integrity, and performance of ETL Pipelines deployed into Azure cloud environments.The role involves developing and automating comprehensive test cases to validate data flows,...

  • Senior Data Pipeline Specialist

    3 days ago

    Bareilly, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    Job DescriptionWe are seeking a skilled Data Engineer to join our team. The ideal candidate will be responsible for designing, building, and maintaining efficient data pipelines.The successful candidate will work closely with cross-functional teams to ensure data is clean, reliable, and accessible for analytics, reporting, and product innovation.Key Skills...

  • Data Extraction Specialist

    1 week ago

    Bareilly, Uttar Pradesh, India beBeeScraping Full time ₹ 10,00,000 - ₹ 16,00,000

    Job Title: Data Extraction SpecialistWe are seeking a skilled professional with expertise in web scraping and automation to join our team. As a Senior Python Developer, you will design, develop, and optimize large-scale data extraction solutions using Scrapy.The ideal candidate will have strong knowledge of Python, including experience with libraries such as...

  • Big Data Specialist

    3 days ago

    Bareilly, Uttar Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Opportunity: Big Data SpecialistWe are seeking a highly skilled big data specialist to design, build, and maintain robust data pipelines for processing channel activity data from multiple sources.The ideal candidate will possess strong knowledge of cloud computing, programming languages like Python, and experience in handling data from various sources...

  • High-Performance Data Pipeline Specialist

    1 week ago

    Bareilly, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Databricks Data Engineer Job DescriptionWe are seeking a highly skilled Senior Databricks Data Engineer to design, develop, and maintain scalable and high-performance data pipelines using Databricks and PySpark.The ideal candidate will have strong hands-on experience in Databricks and AWS, with a solid understanding of PySpark, CI/CD pipelines, and...

  • Highly Skilled Data Engineering Specialist

    2 days ago

    Bareilly, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Senior Data Engineer Job DescriptionWe are seeking a highly skilled data engineer to design, develop and maintain data pipelines that extract data from Oracle Symphony via APIs, process and store it in the Databricks Lakehouse platform.This role requires expertise in data integration, ETL/ELT, APIs, and Databricks. The ideal candidate will have experience...

  • AWS Certified Data Engineer

    13 hours ago

    Bareilly, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 25,00,000

    Senior Data Engineering Specialist">Seeking a highly skilled Senior Data Engineering Specialist to join our team of professionals. The ideal candidate will have extensive experience in designing, developing, and maintaining data pipelines and ETL processes using AWS services.About the Role:Design and implement efficient, reusable, and scalable data pipelines...

  • Senior Data Infrastructure Specialist

    6 hours ago

    Bareilly, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    We are seeking a seasoned GCP Data Engineer with 10+ years of experience. As a specialist in building robust data pipelines on the Google Cloud Platform, you will be responsible for designing, developing and maintaining scalable data pipelines for both batch and real-time processing using GCP tools.Key ResponsibilitiesDesigning and developing data pipelines...

  • Data Intelligence Specialist

    7 hours ago

    Bareilly, Uttar Pradesh, India beBeeData Full time ₹ 90,00,000 - ₹ 1,20,00,000

    Data Intelligence SpecialistThis role involves contributing to the development of EduFinance's data infrastructure, utilizing skills in SQL, Python, and Power BI.ResponsibilitiesMaintain and optimize data collection tools used by all departments.Design and implement robust ETL pipelines using Azure Data Factory, Logic Apps, and APIs.Administer the...