Data Engineer 2

1 week ago

Bengaluru, Karnataka, India Yubi Full time ₹ 15,00,000 - ₹ 25,00,000 per year

Data Engineer 2

Position Summary:

As a Data Engineer, you will be part of a highly talented Data Engineering team. Responsible for developing reusable capabilities and tools to automate various types of data processing pipelines. You will be contributing to different stages of data engineering like data acquisition, ingestion, processing, monitoring pipelines and validating data.

Your contribution will be really crucial in keeping various data ingestion and processing pipelines running successfully. Along with ensuring the data points available in the data lake are up to date, valid and usable.

Technology Experience:

  • 3+ years of experience in data engineering.
  • Comfortable and hands on with the Python programming.
  • Strong experience in working with RDBMS and NoSQL systems.
  • Strong experience in working on AWS ecosystem with hands-on experience in working with different AWS components like Airflow, EMR , Redshift, S3, Athena, PySpark etc.
  • Strong experience in developing REST APIs with Python using frameworks like flask, fastapi.
  • Prior experience in working with crawling libraries like BeautifulSoup in Python would be desirable.
  • Proven ability to work with SQL queries, including writing complex queries to retrieve key metrics. Skilled in connecting to, exploring, and understanding upstream data.
  • Experience working with various data lake storage format types and ability to choose it based on the use cases.

Responsibilities:

  • Design and build scalable data pipelines that can handle large volumes of data.
  • Develop ETL/ELT pipelines and extract the data from any upstream sources and sync with the data lakes with the format of parquet, iceberg, delta formats.
  • Optimize and ensure the data pipelines are running successfully and ensure the business continuity.
  • Collaborate with cross functional teams and source all the data required for the business use cases.
  • Stay up-to-date with emerging data technologies and trends to ensure the continuous improvement of our data infrastructure and architecture
  • Follow best practices in data querying and manipulation to ensure data integrity.

  • DataOps Engineer 2

    2 weeks ago

    Bengaluru, Karnataka, India Astar Data Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We're currently hiring for a DataOps Engineer 2 role at Sigmoid Analytics, and based on your background, I thought you might be a great fitWere looking for professionals with:47 years of experienceA strong development background in Java, Python or GolangHands-on experience with cloud platforms like AWS, Azure, or GCPSolid understanding of DevOps tools...

  • Data Engineer

    4 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    As a Data Engineer, your main responsibilities will involve: The project is a migration of a SAS code to DBT code, so it's important that the candidate has experience in this type of projects. Analyze data models and derive logical conclusions. Processes modeling. Hands on development and monitoring of the Azure cloud Platform and various associated...

  • Data Entry Operator

    2 weeks ago

    Bengaluru, Karnataka, India Design Engineer Full time ₹ 2,64,000 per year

    We are seeking a dedicated and detail-oriented Data Entry Operator (DEO) to support R&D projects. The role involves accurate data entry, documentation, and maintenance of research records in secure systems. The DEO will assist scientists, engineers, and administrative staff by ensuring timely and error-free handling of project information.Key...

  • Data & AI Engineer Lead

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Framework Design & Architecture Architect a metadata-driven, Python/Spark-based framework for automated data validation across high-volume production datasets. Define DQ rule templates for completeness, integrity, conformity, accuracy, and timeliness. Establish data quality thresholds, escalation protocols, and exception workflows. Automation & Integration...

  • Data Engineer

    4 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 8,00,000 - ₹ 16,00,000 per year

    Build scalable data pipelines from structured and unstructured sources Design ontologies that model business entities, relationships, and process flows Collaborate with domain SMEs and ML engineers to structure agent inputs and outputs Implement transformation, validation, and enrichment logic across diverse datasets Maintain and optimize cloud data...

  • Data Engineer

    4 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Migrate ETL workflows from SAP BODS to AWS Glue/dbt/Talend. Develop and maintain scalable ETL pipelines in AWS. Write PySpark scripts for large-scale data processing. Optimize SQL queries and transformations for AWS PostgreSQL. Work with Cloud Engineers to ensure smooth deployment and performance tuning. Integrate data pipelines with existing Unix systems....

  • Data Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Req ID: 338632NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer (HRIS,Tableau and BI) to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Job Duties: Key...

  • Data Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time US$ 90,000 - US$ 1,20,000 per year

    Req ID: 338630NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer (HRIS) to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Job Duties: Key...

  • Data Engineer

    4 days ago

    Bengaluru, Karnataka, India NTT DATA North America Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer - DBT to join our team in Bangalore, Karnātaka (IN-KA), India (IN)."Job Duties: We are seeking an Engineer to...

  • Technical Lead- Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Astar Data Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Kindly find the Job Description Below.Job Title:Technical Lead- Data EngineerLocation: BangaloreYears of Experience: 8+ years of experienceSigmoidworks with a variety of clients from start-ups to fortune 500 companies. We are looking for a detailed oriented self-starter to assist our engineering and analytics teams in various roles as a Software...