Data Engineer 2

24 hours ago

Bangalore, India Yubi Full time

Data Engineer 2

Position Summary:

As a Data Engineer, you will be part of a highly talented Data Engineering team. Responsible for developing reusable capabilities and tools to automate various types of data processing pipelines. You will be contributing to different stages of data engineering like data acquisition, ingestion, processing, monitoring pipelines and validating data.

Your contribution will be really crucial in keeping various data ingestion and processing pipelines running successfully. Along with ensuring the data points available in the data lake are up to date, valid and usable.

Technology Experience:

  • 3+ years of experience in data engineering.
  • Comfortable and hands on with the Python programming.
  • Strong experience in working with RDBMS and NoSQL systems.
  • Strong experience in working on AWS ecosystem with hands-on experience in working with different AWS components like Airflow, EMR , Redshift, S3, Athena, PySpark etc.
  • Strong experience in developing REST APIs with Python using frameworks like flask, fastapi.
  • Prior experience in working with crawling libraries like BeautifulSoup in Python would be desirable.
  • Proven ability to work with SQL queries, including writing complex queries to retrieve key metrics. Skilled in connecting to, exploring, and understanding upstream data.
  • Experience working with various data lake storage format types and ability to choose it based on the use cases.

Responsibilities:

  • Design and build scalable data pipelines that can handle large volumes of data.
  • Develop ETL/ELT pipelines and extract the data from any upstream sources and sync with the data lakes with the format of parquet, iceberg, delta formats.
  • Optimize and ensure the data pipelines are running successfully and ensure the business continuity.
  • Collaborate with cross functional teams and source all the data required for the business use cases.
  • Stay up-to-date with emerging data technologies and trends to ensure the continuous improvement of our data infrastructure and architecture
  • Follow best practices in data querying and manipulation to ensure data integrity.

  • Data Engineer 2

    1 minute ago

    Bangalore, India Yubi Full time

    Data Engineer 2 Position Summary: As a Data Engineer, you will be part of a highly talented Data Engineering team. Responsible for developing reusable capabilities and tools to automate various types of data processing pipelines. You will be contributing to different stages of data engineering like data acquisition, ingestion, processing, monitoring...

  • Data Engineer 2

    15 hours ago

    Bangalore Urban, India Yubi Full time

    Data Engineer 2Position Summary: As a Data Engineer, you will be part of a highly talented Data Engineering team. Responsible for developing reusable capabilities and tools to automate various types of data processing pipelines. You will be contributing to different stages of data engineering like data acquisition, ingestion, processing, monitoring pipelines...

  • Data Scientist-2

    2 hours ago

    bangalore, India Cashfree Payments Full time

    Cashfree Payments is India’s leading payments and API banking company. It provides full-stack payment solutions for businesses in India to accept payments and make payouts via 180+ payment modes - all with a simple integration. Over 6,00,000+ Businesses trust Cashfree Payments with payment collections, vendor payouts, salary payouts, bulk refunds, expense...

  • Data Engineer

    2 minutes ago

    Bangalore, India Us Technology International Private Limited Full time

    Role: Data Engineer Experience 3 -6 years Location: Bangalore 1.Bachelor’s or master’s degree in computer science, Data Science, Engineering, or a related field. 2.Proven experience in predictive modeling using Python. 3.Strong analytical and problem-solving skills. 4.Proficiency in MATLAB and Databricks. 5.Familiarity with Python libraries like...

  • Data Engineer

    23 hours ago

    Bangalore, India Us Technology International Private Limited Full time

    Role: Data Engineer Experience 3 -6 years Location: Bangalore 1.Bachelor’s or master’s degree in computer science, Data Science, Engineering, or a related field. 2.Proven experience in predictive modeling using Python. 3.Strong analytical and problem-solving skills. 4.Proficiency in MATLAB and Databricks. 5.Familiarity with Python libraries...

  • Data Engineer

    2 hours ago

    bangalore, India Us Technology International Private Limited Full time

    Role: Data EngineerExperience 3 -6 yearsLocation: Bangalore1.Bachelor’s or master’s degree in computer science, Data Science, Engineering, or a related field. 2.Proven experience in predictive modeling using Python. 3.Strong analytical and problem-solving skills. 4.Proficiency in MATLAB and Databricks. 5.Familiarity with Python libraries like Matplotlib...

  • Data Engineer

    23 hours ago

    Bangalore, India Geektrust Full time

    Role overview 14-18 Years of experience Experience Experience with native and third-party Databricks integrations (Delta Live Tables, Auto Loader, Databricks Workflows / Apache Airflow, Unity Catalog) 8+ years data engineering experience 3+ years in a senior or principal engineer role Experience designing scalable data pipelines Experience...

  • Data Engineer

    24 hours ago

    Bangalore, India Recro Full time

    Role - Data Engineer Location - Bangalore Experience - 2-4 Yrs Key Responsibilities Design, build, and optimize scalable data pipelines for batch and real-time data processing. Develop and maintain ETL (Extract, Transform, Load) processes to integrate data from multiple sources. Manage and optimize data storage solutions (data warehouses,...

  • Data Scientist

    5 hours ago

    bangalore, India M-Squared | M^2 Full time

    Company DescriptionM-Squared is a marketing attribution community and marketplace that provides businesses with the tools and resources they need to tackle attribution challenges. Our services help businesses gain insights into marketing channel performance, identify revenue-driving channels, and optimize marketing spend. M-Squared offers consultations,...

  • Data Engineer

    3 hours ago

    bangalore, India Recro Full time

    Role - Data EngineerLocation - BangaloreExperience - 2-4 YrsKey ResponsibilitiesDesign, build, and optimize scalable data pipelines for batch and real-time data processing.Develop and maintain ETL (Extract, Transform, Load) processes to integrate data from multiple sources.Manage and optimize data storage solutions (data warehouses, data lakes, relational...