Data Pipeline Engineer

1 day ago


Bengaluru, Karnataka, India beBeeData Full time ₹ 22 - ₹ 28
Job Opportunity

We are seeking a skilled professional to fill the role of Big Data Specialist. This position requires expertise in designing, developing, and maintaining large-scale data pipelines using AWS services such as Glue, Lambda, and DMS.

The ideal candidate will have hands-on experience with Apache Spark (Py Spark) for processing and transforming big datasets, Amazon RDS and relational databases for ingesting and exporting data, and SQL for writing efficient queries.

Key Responsibilities:
  • Design and develop scalable ETL workflows using AWS Glue and Py Spark
  • Collaborate with cross-functional teams to drive data-driven decision-making
  • Develop and maintain complex data models and database designs
  • Write high-quality, well-documented code that follows best practices
  • Monitor and optimize data pipelines for performance and reliability
Requirements:
  • Proficiency in Python, SQL, and AWS services such as Glue, Lambda, and RDS
  • Strong knowledge of Spark, preferably Py Spark, and its applications in big data processing
  • Familiarity with data modeling, database design, and data architecture
  • Excellent problem-solving skills and ability to communicate technical ideas effectively

This is an exciting opportunity to work with cutting-edge technologies and make a significant impact on our organization's data strategy.



  • Bengaluru, Karnataka, India beBeeData Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    The Role of Senior Data Integration SpecialistJob Overview:We are seeking an experienced Senior Data Integration Specialist to join our team. This is a key position that will play a vital role in the smooth operation of our enterprise data integration platform.As a Senior Data Integration Specialist, you will be responsible for designing, implementing, and...


  • Bengaluru, Karnataka, India IDESLABS PRIVATE LIMITED Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    PF Detection is mandatory : Managing data storage solutions on AWS, such as Amazon S3, Amazon Redshift, and Amazon DynamoDB. Implementing and optimizing data processing workflows using AWS services like AWS Glue, Amazon EMR,and AWS Lambda. Working with Spotfire Engineers and business analysts to ensure data is accessible and usable for analysisand...


  • Bengaluru, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Key Roles and Responsibilities:Design, develop, and maintain scalable data pipelines to automate data provision, testing, and monitoring, enhancing the overall efficiency of our CI/CD processes.Collaborate with cross-functional teams to create optimal data pipeline architectures for data science products, ensuring seamless integration and efficient...

  • Pipeline Engineer

    1 week ago


    Bengaluru, Karnataka, India Total Infrastructure Management Solutions Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Job Description: Pipeline Engineer 1We are currently seeking highly motivatedPipeline Engineers (Min 1 Year to 5+ Years)with experience in transmission/ distribution oil and gas pipelines for our consultancy specializing in asset integrity and MAOP (Maximum Allowable Operating Pressure) validation and who can communicate clearly with US-based clients and...


  • Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 26,00,000

    Job Title: Senior Data EngineerDescription:We are seeking a highly skilled data engineer to design, develop, and maintain large-scale data pipelines that extract data from Oracle Symphony via APIs, process and store it in the Databricks Lakehouse platform, and then integrate it into Oracle EPM for financial planning, reporting, and analytics.Key...


  • Bengaluru, Karnataka, India beBeeData Full time ₹ 12,00,000 - ₹ 18,00,000

    Data Engineer Role SummaryWe are seeking a skilled Data Engineer to build and maintain large-scale data pipelines for slot game data. The ideal candidate will have experience with SQL, Python, or other data engineering languages, as well as familiarity with cloud platforms (AWS) and data pipeline orchestration tools.Key ResponsibilitiesDesign, develop, and...


  • Bengaluru, Karnataka, India Steady Rabbit Full time US$ 1,50,000 - US$ 2,00,000 per year

    Experience: 6+ yearsWork Location: BangaloreShift - 1 to 10pm ISTJob Summary:We are looking for a highly skilledAWS Data Engineer with strong experience in data architecture, DevOps practices, and code quality tools like SonarQube. You will be responsible for designing scalable data storage and access patterns using AWS services such asDynamo DB, Redis,...


  • Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineering OpportunityWe are seeking a skilled and motivated Data Engineer to design, develop, and maintain scalable data pipelines that process and transform large volumes of structured and unstructured data.This is an exciting opportunity to work with our team at the forefront of data engineering. If you have a strong background in data engineering,...


  • Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 2,01,20,000

    About this RoleWe are seeking a seasoned data engineer to design and develop scalable data pipelines. As an expert in building data lakes, warehouses, and pipelines, you will play a critical role in orchestrating data workflows and ensuring seamless integration across data services.Your Key Responsibilities:Design and develop data pipelines using Oracle Data...


  • Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job TitleWe are seeking a skilled Data Pipeline Developer to join our team. As a key member of our engineering group, you will be responsible for designing, building, and maintaining data pipelines that drive business success.About the RoleThis is an exciting opportunity to work with cutting-edge technology and collaborate with cross-functional teams. Your...