Data Engineering Senior Specialist

4 weeks ago

PimpriChinchwad, India Telstra Full time
Employment TypePermanent

Closing Date30 Dec 2023 11:59pm

Job TitleData Engineering Senior Specialist - Python & Pyspark

Job Summary

As a Data Platform Engineer Senior Specialist, you bridge the hardware-software boundary. You create, optimise and maintain data storages and platforms so that our users can access high quality and reliable data assets and solutions. You play a key role in transforming epics into delivering new capability/product features for our customers.Job Description

Telstra is Australia's leading telecommunications and technology company with a rich heritage that’s been built over 100 years. From our humble beginnings in the Postmaster General’s Office to the global business we are today our people have been at the forefront of technology innovation. More recently, we have the largest in Australia and are And this is just the beginning of what we’re hoping to achieve together. 

  

We offer a full range of services and compete in all telecommunications markets throughout Australia and are the most well-known brand in technology and communications industry. 

We have operations in more than 20 countries, including in India. In India we are a licensed Telecom Service provider (TSP) and have extended our global networks into India with offices in Bangalore, Mumbai and Delhi. We’ve opened an Innovation and Capability Centre (ICC) in Bangalore, and have a presence in Pune and Hyderabad. In India, we’ve set out to build a platform for innovative delivery and engagement that will strengthen our position as an industry leader. We’re combining innovation, automation and technology to solve the world’s biggest technological challenges in areas such as Internet of Things (IoT), 5G, Artificial Intelligence (AI), Machine Learning, and more. 

  

We’re growing, fast, and for you that means many exciting opportunities to develop your career at Telstra. Join us on this exciting journey, and together, we’ll reimagine the future. 

   

Being part of Data Engineering means you'll be part of a team that focuses on extending our network superiority to enable the continued execution of our digital strategy. With us, you'll be working with world-leading technology and change the way we do IT to ensure business needs drive priorities, accelerating our digitisation programme.  

We are seeking a highly skilled Data Engineer with expertise in Spark, Python, Scala. The successful candidate will be responsible for will be responsible for designing, developing, and maintaining data pipelines using Spark, Python, Scala, and related technologies. The Data Engineer will also be responsible for ensuring data quality, data security, and optimal performance of the data pipelines. Any new engineer would be mostly into developing reusable data processing and storage frameworks that can be used across data platform. 

Hadoop Data Engineering – Hive, Oozie, Yarn / MapReduce, Spark (must) with SCALA (Pref) or Python, Strong SQL 

Job Location – Bangalore or Hyderabad or Pune.

Key Responsibilities

Senior Data Engineer role is to coordinate, and execute all activities related to the requirements interpretation, design and implementation of Data Analytics applications. This individual will apply proven industry and technology experience as well as communication skills, problem-solving skills, and knowledge of best practices to issues related to design, development, and deployment of mission-critical systems with a focus on quality application development and delivery.

This role is key to the success of the Data Engineering capability at Telstra and will be responsible and accountable for the following:

  • Lead the design, development, and maintenance of data pipelines using Spark, Python, Scala, and related technologies.
  • Work with high volume data and ensure data quality and accuracy
  • Implement data security and privacy best practices to protect sensitive data
  • Collaborate with data scientists and business stakeholders to understand data needs and requirements
  • Develop and maintain documentation on data pipeline architecture, data models, and data workflows
  • Mentor and provide technical guidance to junior team members
  • Monitor and troubleshoot data pipelines to ensure they are performing optimally
  • Stay up-to-date with the latest developments in Azure, AWS, Spark, Python, Scala, and related technologies and apply them to solve business problems
  • Optimize data pipelines for cost and performance
  • Automate data processing tasks and workflows to reduce manual intervention.
  • Ability to work in Agile Feature teams
  • Provide training and educate other team members around core capabilities and helps them deliver high quality solutions and deliverables/documentation
  • Self-Motivator to perform Design / Develop user requirements, test and deploy the changes into production

Technical Skills

  • Hands-on experience in the following on Spark Core, Spark SQL, SQL/Hive/Impala
  • Exposure on Hadoop Ecosystem(HDP/Cloudera/MapR/EMR etc) 
  • Experience of working on File formats (Parquet/ORC/AVRO/Delta/Hudi etc.) 
  • Experience with high volume data processing and data streaming technologies
  • Experience of using Orchestration tools like Control-m,Azure Data Factory,Airflow,Luigi to schedule jobs.
  • Demonstrated experience leading data engineering projects and mentoring junior team members.
  • Data engineer with expertise of working on Azure Cloud using Databricks, Kinesis/ Azure Event hub Flume/Kafka/Spark streaming, Azure Data Factory.
  • Strong experience in data modeling, schema design, and ETL development using SQL and related technologies
  • Familiarity with data security and privacy best practices
  • Good exposure on TDD
  • Exposure on using CI tools like Git, Bitbucket, Github, Gitlab, Azure DevOps
  • Exposure on using CD tools like Jenkins, Bamboo, Azure DevOps
  • Exposure on Observability tools like Azure Monitor, Graphana etc
  • Prior experience in building or working in team building reusable frameworks
  • Good understanding of Data Achitecture and design principles. (Delta/Kappa/Lambda architecture) 
  • Exposure to Code Quality - Static and Dynamic code scans 
  • Experience in designing solutions for multiple large data warehouses with a good understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS and/or knowledge on NoSQL platforms.
  • Should be able to provided scalable and robust solution architecture depending on the business needs.
  • Propose best practices/standards
  • Programming & Databases - Java /Python/Scala/ SQL Procedure. Multi tenanted databases / Spark

  • Data Engineering Senior Specialist

    4 weeks ago

    pimpri-chinchwad, India Telstra Full time

    Employment Type Permanent Closing Date 30 Dec 2023 11:59pm Job Title Data Engineering Senior Specialist - Python & Pyspark Job Summary As a Data Platform Engineer Senior Specialist, you bridge the hardware-software boundary. You create, optimise and maintain data storages and platforms so that our users can access high quality and reliable data assets...

  • Senior Data Engineer

    2 weeks ago

    Pimpri-Chinchwad, India Lear Corporation Full time

    Overview:As aSenior Data Engineerat Lear, you will take a leadership role in designing, building, and maintaining robust data pipelines within the Foundry platform. Your expertise will drive the seamless integration of data and analytics, ensuring high-quality datasets and supporting critical decision-making processes. If you’re passionate about data...

  • Senior Data Engineer

    2 weeks ago

    Pimpri-Chinchwad, India Lear Corporation Full time

    Overview:As a Senior Data Engineer at Lear, you will take a leadership role in designing, building, and maintaining robust data pipelines within the Foundry platform. Your expertise will drive the seamless integration of data and analytics, ensuring high-quality datasets and supporting critical decision-making processes. If you’re passionate about data...

  • Senior Software Engineer

    4 weeks ago

    Pimpri-Chinchwad, India RS Consulting Full time

    We are hiring for Senior Software engineers (full stack developers) for a German software company located in Baner, Pune.Experience: 5 to 8 yearsJob Location: Hinjewadi, PuneKey Skills: C#,.Net, React, SQL Server, JavaScript, TypeScript, HTML, CSS, Azure

  • Senior Software Engineer

    4 weeks ago

    Pimpri-Chinchwad, India RS Consulting Full time

    We are hiring for Senior Software engineers (full stack developers) for a German software company located in Baner, Pune.Experience: 5 to 8 yearsJob Location: Hinjewadi, PuneKey Skills: C#,.Net, React, SQL Server, JavaScript, TypeScript, HTML, CSS, Azure

  • Senior Data Engineer

    2 weeks ago

    Pune/Pimpri-Chinchwad Area, India Lear Corporation Full time

    Overview:As a Senior Data Engineer at Lear, you will take a leadership role in designing, building, and maintaining robust data pipelines within the Foundry platform. Your expertise will drive the seamless integration of data and analytics, ensuring high-quality datasets and supporting critical decision-making processes. If you’re passionate about data...

  • Senior Software Engineer

    3 weeks ago

    Pimpri-Chinchwad, India ConnectWise Full time

    Senior Software Engineer (Dot Net) Mumbai/PuneExperience – 5 to 10 years.Job type – Permanent, HybridJob location – Pune & MumbaiJOB DESCRIPTION Required Skills: • Programming languages: o ASP.net, .NET core, C#, JavaScript, SQL, MVC, Web API, o Experience with Intellij / Eclipse, Visual Studio and Git Source Control o Experience with HTML, CSS,...

  • Data Engineering Developer

    4 weeks ago

    pimpri-chinchwad, India Telstra Full time

    Employment Type Permanent Closing Date 29 Sept 2023 11:59pm Job Title Data Engineering Developer - Python & Pyspark Job Summary As a Data Management Analyst, you value and steward strong governance, architecture and data exchange. You play an important role in supporting the implementation of effective data management strategies to ensure that Telstra’s...

  • Data Engineering Developer

    4 weeks ago

    Pimpri-Chinchwad, India Telstra Full time

    Employment TypePermanentClosing Date29 Sept 2023 11:59pmJob TitleData Engineering Developer - Python & PysparkJob SummaryAs a Data Management Analyst, you value and steward strong governance, architecture and data exchange. You play an important role in supporting the implementation of effective data management strategies to ensure that Telstra’s data is...

  • Senior Engineer Quality Management

    1 week ago

    Pimpri-Chinchwad, India MAHLE Full time

    Work Experience RequirementsRole and Responsibilities:- Min 3 Years in similar role or experience with customer & supplier Quality management- Prepares and evaluates Production Part Approval Process (PPAP) documents for internal customers and suppliers- Ability to read understand and assess engineering drawings, CAD data and Geometric dimensions &...

  • Senior Engineer Quality Management

    3 weeks ago

    pimpri-chinchwad, India MAHLE Full time

    Work Experience RequirementsRole and Responsibilities:Min 3 Years in similar role or experience with customer & supplier Quality managementPrepares and evaluates Production Part Approval Process (PPAP) documents for internal customers and suppliersAbility to read understand and assess engineering drawings, CAD data and Geometric dimensions & tolerancing.Lead...

  • Senior Engineer Quality Management

    4 weeks ago

    Pimpri Chinchwad, India MAHLE Full time

    Work Experience RequirementsRole and Responsibilities:Min 3 Years in similar role or experience with customer & supplier Quality managementPrepares and evaluates Production Part Approval Process (PPAP) documents for internal customers and suppliersAbility to read understand and assess engineering drawings, CAD data and Geometric dimensions & tolerancing.Lead...

  • Senior Engineer Quality Management

    4 weeks ago

    Pimpri-Chinchwad, India MAHLE Full time

    Work Experience RequirementsRole and Responsibilities:Min 3 Years in similar role or experience with customer & supplier Quality managementPrepares and evaluates Production Part Approval Process (PPAP) documents for internal customers and suppliersAbility to read understand and assess engineering drawings, CAD data and Geometric dimensions & tolerancing.Lead...

  • Senior Engineer Quality Management

    3 weeks ago

    Pimpri Chinchwad, India MAHLE Full time

    Work Experience Requirements Role and Responsibilities: Min 3 Years in similar role or experience with customer & supplier Quality management Prepares and evaluates Production Part Approval Process (PPAP) documents for internal customers and suppliers Ability to read understand and assess engineering drawings, CAD data and Geometric dimensions &...

  • Senior Design Engineer

    3 weeks ago

    Pimpri-Chinchwad, India Crane India Full time

    The ideal candidate will be responsible to carry out design activities of NPD, VAVE and Sustaining Engineering projects under supervision.Essential Qualifications & Experience:Mechanical Engineering Degree from reputed engineering college with 4-8 years' experience in R&D/ engineering preferably in Mechanical Design and development of Industrial Valves.Hands...

  • Senior Design Engineer

    3 weeks ago

    Pimpri-Chinchwad, India Crane India Full time

    The ideal candidate will be responsible to carry out design activities of NPD, VAVE and Sustaining Engineering projects under supervision.Essential Qualifications & Experience:Mechanical Engineering Degree from reputed engineering college with 4-8 years' experience in R&D/ engineering preferably in Mechanical Design and development of Industrial Valves.Hands...

  • Senior Firmware Engineer

    3 weeks ago

    Pimpri-Chinchwad, India UST Full time

    UST ( is looking for Firmware Engineer for Pune Location.Experience : 10+ YearsJob Description:• BE/BTech/ME/Mtech in Electronics/Electrical/Telecomm/Instrumentation• Minimum 10~12 years of experience in Embedded Firmware design & development• Experience in performing activities listed in deliverables slides• Knowledge 32-bit ARM Mx microcontrollers...

  • Senior Software Engineer

    4 weeks ago

    Pune/Pimpri-Chinchwad Area, India RS Consulting Full time

    We are hiring for Senior Software engineers (full stack developers) for a German software company located in Baner, Pune.Experience: 5 to 8 yearsJob Location: Hinjewadi, PuneKey Skills: C#,.Net, React, SQL Server, JavaScript, TypeScript, HTML, CSS, Azure

  • Senior Software Engineer

    4 weeks ago

    Pune/Pimpri-Chinchwad Area, India RS Consulting Full time

    We are hiring for Senior Software engineers (full stack developers) for a German software company located in Baner, Pune. Experience: 5 to 8 years Job Location: Hinjewadi, Pune Key Skills: C#,.Net, React, SQL Server, JavaScript, TypeScript, HTML, CSS, Azure

  • Senior Firmware Engineer

    3 weeks ago

    Pimpri-Chinchwad, India UST Full time

    UST (www.ust.com) is looking for Firmware Engineer for Pune Location.Experience : 10+ YearsJob Description:• BE/BTech/ME/Mtech in Electronics/Electrical/Telecomm/Instrumentation• Minimum 10~12 years of experience in Embedded Firmware design & development• Experience in performing activities listed in deliverables slides• Knowledge 32-bit ARM Mx...