Data Engineer 2

1 day ago

Bangalore Urban, India Yubi Full time

Data Engineer 2

Position Summary:

As a Data Engineer, you will be part of a highly talented Data Engineering team. Responsible for developing reusable capabilities and tools to automate various types of data processing pipelines. You will be contributing to different stages of data engineering like data acquisition, ingestion, processing, monitoring pipelines and validating data.

Your contribution will be really crucial in keeping various data ingestion and processing pipelines running successfully. Along with ensuring the data points available in the data lake are up to date, valid and usable.

Technology Experience:

  • 3+ years of experience in data engineering.
  • Comfortable and hands on with the Python programming.
  • Strong experience in working with RDBMS and NoSQL systems.
  • Strong experience in working on AWS ecosystem with hands-on experience in working with different AWS components like Airflow, EMR , Redshift, S3, Athena, PySpark etc.
  • Strong experience in developing REST APIs with Python using frameworks like flask, fastapi.
  • Prior experience in working with crawling libraries like BeautifulSoup in Python would be desirable.
  • Proven ability to work with SQL queries, including writing complex queries to retrieve key metrics. Skilled in connecting to, exploring, and understanding upstream data.
  • Experience working with various data lake storage format types and ability to choose it based on the use cases.

Responsibilities:

  • Design and build scalable data pipelines that can handle large volumes of data.
  • Develop ETL/ELT pipelines and extract the data from any upstream sources and sync with the data lakes with the format of parquet, iceberg, delta formats.
  • Optimize and ensure the data pipelines are running successfully and ensure the business continuity.
  • Collaborate with cross functional teams and source all the data required for the business use cases.
  • Stay up-to-date with emerging data technologies and trends to ensure the continuous improvement of our data infrastructure and architecture
  • Follow best practices in data querying and manipulation to ensure data integrity.

  • Data Engineer 2

    11 hours ago

    Bangalore, India Yubi Full time

    Data Engineer 2 Position Summary: As a Data Engineer, you will be part of a highly talented Data Engineering team. Responsible for developing reusable capabilities and tools to automate various types of data processing pipelines. You will be contributing to different stages of data engineering like data acquisition, ingestion, processing, monitoring...

  • Data Engineer

    8 hours ago

    Bangalore Urban, India Us Technology International Private Limited Full time

    Role: Data Engineer Experience 3 -6 years Location: Bangalore 1.Bachelor’s or master’s degree in computer science, Data Science, Engineering, or a related field. 2.Proven experience in predictive modeling using Python. 3.Strong analytical and problem-solving skills. 4.Proficiency in MATLAB and Databricks. 5.Familiarity with Python libraries like...

  • Data Engineer

    1 day ago

    Bangalore Urban, India Us Technology International Private Limited Full time

    Role: Data EngineerExperience 3 -6 yearsLocation: Bangalore1.Bachelor’s or master’s degree in computer science, Data Science, Engineering, or a related field. 2.Proven experience in predictive modeling using Python. 3.Strong analytical and problem-solving skills. 4.Proficiency in MATLAB and Databricks. 5.Familiarity with Python libraries like Matplotlib...

  • Data Engineer 2

    1 day ago

    Bangalore, India Yubi Full time

    Data Engineer 2 Position Summary: As a Data Engineer, you will be part of a highly talented Data Engineering team. Responsible for developing reusable capabilities and tools to automate various types of data processing pipelines. You will be contributing to different stages of data engineering like data acquisition, ingestion, processing, monitoring...

  • Software Engineer Backend

    1 day ago

    Bangalore Urban, India Arcana Full time

    Join our dynamic team at the forefront of cutting-edge technology as we seek a SDE2 (Backend). Embark on a journey where your deep-rooted expertise in computer science fundamentals, alongside an intricate understanding of data structures, algorithms, and system design, becomes the cornerstone of innovative solutions. This pivotal role not only demands your...

  • Senior Principal Data Engineer

    1 day ago

    Bangalore Urban, India MakeMyTrip Full time

    Position: Senior Principal Data EngineerExperience: Must have 10+ years of experienceAbout Role:We are looking for experienced Data engineers with excellent problem-solving skills to develop machine-learning powered Data Products design to enhance customer experiences.About us:Nurtured from the seed of a single great idea - to empower the traveler -...

  • Senior Principal Data Engineer

    8 hours ago

    Bangalore Urban, India MakeMyTrip Full time

    Position: Senior Principal Data Engineer Experience: Must have 10+ years of experience About Role: We are looking for experienced Data engineers with excellent problem-solving skills to develop machine-learning powered Data Products design to enhance customer experiences. About us: Nurtured from the seed of a single great idea - to empower the traveler...

  • Data Engineer

    8 hours ago

    Bangalore Urban, India Synechron Full time

    Good-day, We have immediate opportunity for Data Engineer (Python, Spark/Scala) . Job Role: Data Engineer (Python, Spark/Scala) Job Location: Bangalore Experience- 5 to 10 years Notice Period: Immediate joiner only. About Company: At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm...

  • Data Engineer

    1 day ago

    Bangalore Urban, India Synechron Full time

    Good-day,We have immediate opportunity for Data Engineer (Python, Spark/Scala).Job Role: Data Engineer (Python, Spark/Scala)Job Location: Bangalore Experience-5 to 10 yearsNotice Period: Immediate joiner only.About Company:At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity...

  • Senior Data Engineer

    8 hours ago

    Bangalore Urban, India Tata Consultancy Services Full time

    Greetings from TCS!!! TCS has been a great pioneer in feeding the fire of Techies like you. We are a global leader in the technology arena and there’s nothing that can stop us from growing together. Your role is of key importance, as it lays down the foundation for the entire project. Make sure you have a valid EP number before the interview. To create an...