Python Data Engineer

3 weeks ago

Noida India ShyftLabs Full time

Job Description

Position Overview

We are seeking a highly skilled Python Data Engineer with a strong background in designing, developing, and maintaining scalable data solutions. The ideal candidate will have 5+ years of professional experience in Python and SQL, with proven expertise in building ETL/ELT pipelines, managing data streaming solutions. This role requires a hands-on engineer who can translate business requirements into reliable data systems, optimize performance, and ensure high data quality.

ShyftLabs is a growing data product company that was founded in early 2020 and works primarily with Fortune 500 companies. We deliver digital solutions built to help accelerate the growth of businesses in various industries, by focusing on creating value through innovation.

Job Responsibilities

- Develop and maintain data pipelines and ETL/ELT processes using Python
- Design and implement scalable, high-performance applications
- Work collaboratively with cross-functional teams to define requirements and deliver solutions
- Develop and manage near real-time data streaming solutions using PubSub or Beam
- Contribute to code reviews, architecture discussions, and continuous improvement initiatives
- Monitor and troubleshoot production systems to ensure reliability and Performance

Basic Qualification

- 5+ years of professional software development experience with Python and SQL
- Strong understanding of software engineering best practices (testing, version control, CI/CD)
- Experience building and optimizing ETL/ELT processes and data pipelines
- Proficiency with SQL and database concepts
- Experience with data processing frameworks (e.g., Pandas)
- Understanding of software design patterns and architectural principles
- Ability to write clean, well-documented, and maintainable code
- Experience with unit testing and test automation
- Experience working with any cloud provider (GCP is preferred)
- Experience with CI/CD pipelines and Infrastructure as code
- Experience with Containerization technologies like Docker or Kubernetes

Preferred Qualification

- Experience with GCP services, particularly Cloud Run and Dataflow
- Experience with stream processing technologies (Pub/Sub)
- Familiarity with big data technologies (Airflow)
- Experience with data visualization tools and libraries
- Knowledge of CI/CD pipelines with Gitlab and infrastructure as code with Terraform
- Familiarity with platforms like Snowflake, Bigquery or Databricks,

We are proud to offer a competitive salary alongside a strong insurance package. We pride ourselves on the growth of our employees, offering extensive learning and development resources.

  • 3 Days Left: Python developer

    3 weeks ago

    Noida, India Gauge data solutions Full time

    Job Description Essential Skills - Develop, enhance and maintain Python related projects, data services, platforms and processes. - Apply and maintain data quality checks to ensure data integrity and completeness. - Able to integrate multiple data sources and databases. - Collaborate with cross-functional teams across, Decision Sciences, Search, Database...

  • Data Engineer

    2 weeks ago

    Pune, India Go Digital Technology Consulting LLP Full time

    Job Description Job Title: Python Data Engineer Location: Pune / Remote Experience Level: 38 years Role Overview We are looking for a Python-centric Data Engineer who can design and maintain scalable, high-performance data pipelines for large and complex datasets. The ideal candidate brings strong object-oriented Python programming skills, experience with...

  • Eucloid - Senior/Lead Data Engineer - Python

    2 weeks ago

    Chennai, India Eucloid Data Solutions Full time

    Job Description Job Description Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and...

  • Python developer

    4 days ago

    Noida, Uttar Pradesh, India Gauge data solutions Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    Essential SkillsDevelop, enhance and maintain Python related projects, data services, platforms and processes.Apply and maintain data quality checks to ensure data integrity and completeness.Able to integrate multiple data sources and databases.Collaborate with cross-functional teams across, Decision Sciences, Search, Database Management. To design...

  • Data & AI Engineer Lead

    1 week ago

    India NTT DATA Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Req ID: 344686NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data & AI Engineer Lead to join our team in Remote, Karnātaka (IN-KA), India (IN). "Job Duties: Role Overview The...

  • Python Data Engineer

    4 weeks ago

    Bengaluru, India Capgemini Full time

    Job Description Location:Bangalore (Preferred), Gurgaon, Chennai Experience:7+ years Notice Period:Immediate to 15 days Role Level:B2 / C1 Choosing Capgemini means choosing a place where you'll be empowered to shape your career, supported by a collaborative global community, and inspired to reimagine what's possible. Join us in helping leading...

  • Data engineer

    3 weeks ago

    India Data-Hat AI Full time

    Department: Data Engineering & AI Solutions  Reports To: Lead Data Solutions Architect  Travel: International travel required (up to 30–40%)   Position Summary:   We are hiring a senior-level Data Engineer to lead the design, development, and optimization of high-performance data infrastructure that underpins mission-critical AI systems. With 12+...

  • Lead Data Engineer- Python

    4 hours ago

    Noida, Uttar Pradesh, India Infinium Associates Full time ₹ 40,00,000 - ₹ 48,00,000 per year

    About the Role:We are seeking a talented Lead Data Engineer to join our team and play a pivotal role in transforming raw data into valuable insights. As a Data Engineer, you will design, develop, and maintain robust data pipelines and infrastructure to support our organization's analytics and decision-making processes.Responsibilities:Data Pipeline...

  • Python API + Data Engineer

    3 weeks ago

    India Digitrix Software LLP Full time

    Experience : 5 to 8 years Job description: Python API + Data Engineer We are looking for a self-motivated Python Data Engineer with strong skills in backend/API development and large-scale data processing , who can work independently and thrive in a fast-paced, agile environment. Key Responsibilities Design, develop, and optimize data pipelines using...

  • Python aws data engineer

    4 weeks ago

    India Digitrix Software LLP Full time

    Experience : 5 to 8 years Job description: Python AWS Data Engineer Python, AWS Python (core language skill) -- Backend, Pandas, Py Spark (Data Frame API), interacting with AWS (e.g., boto3 for S3, Glue, Lambda) Data Processing: Spark (Py Spark), Glue, EMR AWS Core Services: S3, Glue, Athena, Lambda, Step Functions, EMR Containerization: Docker ...