Data Engineer for Large-Scale Data Pipelines

3 hours ago

Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 26,00,000

Job Title: Senior Data Engineer

Description:

We are seeking a highly skilled data engineer to design, develop, and maintain large-scale data pipelines that extract data from Oracle Symphony via APIs, process and store it in the Databricks Lakehouse platform, and then integrate it into Oracle EPM for financial planning, reporting, and analytics.

Key Responsibilities:

  • Design and implement end-to-end pipelines from Oracle Symphony (API extraction) into Databricks Lakehouse.
  • Develop efficient ETL/ELT processes in Databricks (PySpark, Delta Lake) to transform, cleanse, and enrich data.
  • Build and maintain data flows from Databricks into Oracle EPM to support reporting, forecasting, and planning.
  • Ensure data quality, consistency, and governance across Symphony, Databricks, and EPM.
  • Optimize pipeline performance, scalability, and reliability.
  • Collaborate with data architects, finance teams, and Oracle specialists to meet business needs.
  • Troubleshoot pipeline issues and provide production support for data integration processes.
  • Document architecture, pipeline logic, and integration workflows.
  • Stay current on Databricks, Oracle, and API integration best practices.

Required Skills & Qualifications:

  • Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.
  • 5+ years of experience in data engineering, ETL/ELT, and data pipeline development.
  • Hands-on experience with Databricks (PySpark, Delta Lake, MLflow).
  • Strong experience with APIs (REST, SOAP, JSON, XML) for data extraction and integration.
  • Proficiency in SQL, Python, and Spark for data processing.
  • Experience with cloud platforms (Azure, AWS, or GCP) for hosting Databricks and related services.
  • Knowledge of data modeling, data governance, and performance tuning.
  • Strong problem-solving skills and ability to work in cross-functional teams.

Candidates must have the ability to:

  • Maintain high standards of data quality and integrity.
  • Demonstrate proficiency in multiple programming languages.
  • Work effectively in fast-paced environments with tight deadlines.

This role requires strong technical expertise, excellent collaboration skills, and the ability to drive results in a dynamic environment. We encourage applications from experienced professionals looking to make a meaningful impact in data engineering.

  • Expert Data Engineering Services for Large-Scale Pipeline Development

    5 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Freelance Data Engineer Opportunity">We are seeking an experienced Freelance Data Engineer with expertise in Python, SQL, Linux/Bash, PySpark, AWS, and Databricks to design, build, and optimize large-scale data pipelines. The consultant will collaborate with internal teams and clients to ensure robust, scalable, and high-performance data solutions.

  • Large Scale Data Systems Engineer

    4 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineer\ Are you a seasoned data engineer with 6-8 years of experience? Do you have a passion for building and maintaining large-scale data systems on AWS?\ Job Description:\ As a senior data engineer, you will play a key role in designing and implementing efficient data pipelines using Spark, PySpark, and Scala. You will work closely with our...

  • Top Data Engineer for Large-Scale Projects

    1 week ago

    Bengaluru, Karnataka, India beBeeSeniorDeveloper Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Are you a highly skilled data engineering professional looking for a new challenge?We are seeking an experienced Senior Developer to join our team and lead the development of large-scale data processing systems on Google Cloud Platform.Job Description:The ideal candidate will have a strong background in data engineering and be proficient in designing,...

  • Large Data Engineer

    4 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 60,00,000 - ₹ 1,50,00,000

    About this Role : We are seeking a seasoned data expert to design, develop, and optimize large-scale data processing systems. This individual will work with high-volume, high-velocity datasets ensuring the platforms scalability, performance, and reliability. This role requires expertise in big data frameworks, distributed systems, and data engineering best...

  • Data Architect for Large-Scale Data Systems

    17 hours ago

    Bengaluru, Karnataka, India beBeeBigData Full time ₹ 20,00,000 - ₹ 25,00,000

    We are seeking a seasoned Data Engineer to develop and maintain large-scale data systems. This role involves designing, building, and deploying data architectures that enable business growth, analytics, and machine learning needs. The ideal candidate will have extensive experience working with big data technologies, including Spark, Java/Scala, and cloud...

  • Urgent Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India NTT Data Full time

    Job DescriptionNTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer - ETL to join our team in Bangalore, Karntaka (IN-KA), India (IN).Job Duties:. Migrate ETL workflows from...

  • Large Scale Data Solutions Architect

    2 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title:">The Data Engineering Lead is responsible for building and managing large-scale data pipelines and data warehousing solutions using Snowflake.">">Key Responsibilities:">">">Design, develop, and maintain scalable data pipelines using Snowflake.">Build and optimize data models, schemas, and data warehouses for performance and efficiency.">Perform...

  • Large Scale Data Solutions Architect

    4 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 18,00,000

    Senior Azure Data Engineer Job DescriptionThe ideal candidate will have hands-on experience with Azure technologies, including Azure Data Factory (ADF), Azure Synapse Analytics, PySpark, Python, Delta Tables, and Unity Catalog.As a Senior Azure Data Engineer, you will be responsible for designing, building, and maintaining large-scale data platforms on...

  • Data Pipeline Architect

    1 day ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 12,00,000 - ₹ 18,00,000

    Data Engineer Role SummaryWe are seeking a skilled Data Engineer to build and maintain large-scale data pipelines for slot game data. The ideal candidate will have experience with SQL, Python, or other data engineering languages, as well as familiarity with cloud platforms (AWS) and data pipeline orchestration tools.Key ResponsibilitiesDesign, develop, and...

  • Large Scale Data Architect

    2 weeks ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Opportunity\ \A Data Systems Designer is required to develop and maintain large-scale data systems. This includes designing scalable data architectures, developing systems on public cloud providers, and working with NoSQL databases and APIs.\