Data Engineer, Pyspark

1 week ago

Bengaluru, Karnataka, India NatWest Group Full time ₹ 1,50,00,000 - ₹ 2,50,00,000 per year

Join us as a Data Engineer

  • We're looking for someone to build effortless, digital first customer experiences to help simplify our organisation and keep our data safe and secure
  • Day-to-day, you'll develop innovative, data-driven solutions through data pipelines, modelling and ETL design while inspiring to be commercially successful through insights
  • If you're ready for a new challenge, and want to bring a competitive edge to your career profile by delivering streaming data ingestions, this could be the role for you
  • We're offering this role at associate vice president level

What you'll do

Your daily responsibilities will include you developing a comprehensive knowledge of our data structures and metrics, advocating for change when needed for product development. You'll also provide transformation solutions and carry out complex data extractions.

We'll expect you to develop a clear understanding of data platform cost levels to build cost-effective and strategic solutions. You'll also source new data by using the most appropriate tooling before integrating it into the overall solution to deliver it to our customers.

You'll Also Be Responsible For

  • Driving customer value by understanding complex business problems and requirements to correctly apply the most appropriate and reusable tools to build data solutions
  • Participating in the data engineering community to deliver opportunities to support our strategic direction
  • Carrying out complex data engineering tasks to build a scalable data architecture and the transformation of data to make it usable to analysts and data scientists
  • Building advanced automation of data engineering pipelines through the removal of manual stages
  • Leading on the planning and design of complex products and providing guidance to colleagues and the wider team when required

The skills you'll need

To be successful in this role, you'll have an understanding of data usage and dependencies with wider teams and the end customer. You'll also have experience of extracting value and features from large scale data.

We'll expect you to have at least seven years of experience in ETL technical design, data quality testing, cleansing and monitoring, data sourcing, exploration and analysis, and data warehousing and data modelling capabilities.

You'll Also Need

  • Experience of using programming languages alongside knowledge of data and software engineering fundamentals
  • Experience in Oracle PL-SQL, PySpark, AWS S3, Glue, and Airflow
  • Good knowledge of modern code development practices
  • Great communication skills with the ability to proactively engage with a range of stakeholders

  • Data Engineer- 1

    1 day ago

    Bengaluru, Karnataka, India Databricks PySpark Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Position: Senior Data Engineer We are seeking an experienced Senior Data Engineer to design and implement high-performance data pipelines, ensuring the smooth flow of large datasets across systems. In this role, you will leverage your expertise in Databricks, Pyspark, and ETL design to build efficient data solutions. You will also contribute to...

  • Oracle + PySpark Data Engineer

    1 day ago

    Bengaluru, Karnataka, India PradeepIT Consulting Services Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Job Description: Experiance:5 to 7 YearsWe are seeking a highly skilled and motivated Oracle + PySpark Data Engineer/Analyst to join our team. The ideal candidate will be responsible for leveraging the Oracle database and PySpark to manage, transform, and analyze data to support our business's decision-making processes. This role will play a crucial part...

  • Data Engineer

    3 days ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    Job Duties:Migrate ETL workflows from SAP BODS to AWS Glue/dbt/Talend. Develop and maintain scalable ETL pipelines in AWS. Write PySpark scripts for large-scale data processing. Optimize SQL queries and transformations for AWS PostgreSQL. Work with Cloud Engineers to ensure smooth deployment and performance tuning. Integrate data pipelines with existing Unix...

  • Data Engineer – Databricks

    4 weeks ago

    Bengaluru, Karnataka, India, Karnataka Capgemini Engineering Full time

    Data Engineer – Databricks & PySparkLocation: BangaloreExperience: 6–9 yearsChoosing Capgemini means choosing a place where you’ll be empowered to shape your career, supported by a collaborative global community, and inspired to reimagine what’s possible. Join us in helping leading Consumer Products and Retail Services (CPRS) organizations unlock the...

  • Sr. Snowflake Data Engineer Pyspark

    2 weeks ago

    Bengaluru, Karnataka, India EduRun Group Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Senior Data Engineer | 10-12 Years Experience | Must Have: Proven expertise in building scalable batch and streaming data pipelines using Databricks (PySpark) and Snowflake.Lead the design, implementation, and optimization of application data stores using PostgreSQL, DynamoDB, and advanced SQL.Strong programming skills in SQL, Python, and PySpark for...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Req ID:321800We are currently seeking a Data Engineer (Talend &Pyspark) to join our team in Bangalore, Karntaka (IN-KA), India (IN)."Job Duties: Key Responsibilities: Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical...

  • Pyspark Lead

    2 days ago

    Bengaluru, Karnataka, India Krazy Mantra HR Solutions Pvt. Ltd Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for a skilled PySpark Lead with 5 to 8 years of experience, based in Bangalore. The ideal candidate will have expertise in PySpark, SQL, ETL/Data Engineering Pipeline, Cloud understanding, CI/CD, Data Modelling, Airflow, Kafka, EMR/Databricks, Snowflake, and Hive.Roles and ResponsibilityDesign and develop scalable data pipelines using PySpark...

  • Python Pyspark

    1 day ago

    Bengaluru, Karnataka, India Tata Consultancy Services (TCS) Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Greetings from Tata Consultancy Services (TCS)Location : Chennai, Bangalore, Pune, DelhiExperience : 5-10 YearsRole: Python Pyspark DeveloperRequired Skill set Python, Pyspark, Scala, SparkMust HaveStrong analytical skills in conducting sophisticated statistical analysis using bureau/vendor data, customer performances.Working experience in languages PySpark...

  • Software Engineer, PySpark

    2 weeks ago

    Bengaluru, Karnataka, India NatWest Group Full time ₹ 2,00,00,000 - ₹ 2,50,00,000 per year

    Software Engineer, PySpark, VP Join us as a Software Engineer, PySparkThis is an opportunity for a technically minded individual to join us as a Software Engineer You'll be designing, producing, testing and implementing working software, working across the lifecycle of the system Hone your existing software engineering skills and advance your career...

  • Data Engineer with SQL/ETL/Pyspark

    5 days ago

    Bengaluru, Karnataka, India Infosys Finacle Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Mandate Skills - SQL, ETL ,hadoop, pyspark, Apache, KafkaRequired Skills5+ years of relevant Data engineering experience using Python or Java as tech be well versed into HadoopDevelopment expertise with 2+ years of experience, in SQL, ETL processes and other similar technologiesSkilled in Identifying and optimizing database queriesStrong understanding of...