Data Engineer for Cloud Infrastructure
3 days ago
Telstra is a leading telecommunications and technology company with a rich heritage built over 100 years. With operations in more than 20 countries, including India, we're combining innovation, automation, and technology to solve the world's biggest technological challenges.
Salary:$130,000 - $180,000 per annum
Job Description
We're seeking a highly skilled Data Engineer with expertise in Spark, Python, Scala to design, develop, and maintain data pipelines using Spark, Python, Scala, and related technologies on Hadoop/Clouderra Platform.
Key Responsibilities
- Design, develop, and maintain data pipelines using Spark, Python, Scala, and related technologies on Hadoop/Clouderra Platform
- Work with high volume data and ensure data quality and accuracy
- Implement data security and privacy best practices to protect sensitive data
- Develop and maintain documentation on data pipeline architecture, data models, and data workflows
- Monitor and troubleshoot data pipelines to ensure they are performing optimally
- Stay up-to-date with the latest developments in Azure, AWS, Spark, Python, Scala, and related technologies and apply them to solve business problems
Required Skills and Qualifications
- Hands-on experience in Spark Core, Spark SQL, SQL/Hive/Impala
- Exposure on Hadoop Ecosystem (HDP/Cloudera/MapR/EMR etc)
- Experience of working on File formats (Parquet/ORC/AVRO/Delta/Hudi etc.)
- Experience with high volume data processing and data streaming technologies
- Experience of using Orchestration tools like Control-m
- Strong experience in data modeling, schema design, and ETL development using SQL and related technologies
- Familiarity with data security and privacy best practices
- Good exposure on TDD
- Exposure on using CI tools like Git, Bitbucket, Github, Gitlab, Azure DevOps
- Exposure on using CD tools like Jenkins, Bamboo, Azure DevOps
- Cloud exposure (Hadoop)
- Experience and knowledgeable on the following: Azure data offerings - ADF, ADLS2, Azure Databricks, Azure Synapse, Eventhubs, CosmosDB etc, Presto/Athena
- Exposure of working on Power BI
- Prior experience in building or working in team building reusable frameworks
- Good understanding of Data Architecture and design principles. (Delta/Kappa/Lambda architecture)
- Exposure to Code Quality - Static and Dynamic code scans
- Good knowledge of NoSQL Databases/ HBase/ MongoDB / Cassandra / Cosmos DB
- Good knowledge of GraphDB(Neo4J)
- Experience with enterprise data management, Datawarehouse, data modelling, Business Intelligence, data integration
- Expertise in SQL and stored procedure
- Experience in designing solutions for multiple large data warehouses with a good understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS and/or knowledge on NoSQL platforms
- Experience in working on Azure SQL Data warehouse (Synapse) and Azure analysis service or Redshift or Synapse
-
Data Engineering Developer
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India Telstra Full timeJob TitleData Engineering DeveloperAbout the RoleWe are seeking a highly skilled Data Engineer with expertise in Spark, Python, and Scala to join our team at Telstra. As a Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines using Spark, Python, Scala, and related technologies.Key ResponsibilitiesDesign, develop,...
-
Data Engineering Senior Specialist
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India Telstra Full timeJob TitleData Engineering Senior SpecialistJob SummaryAs a Data Platform Engineer Senior Specialist, you bridge the hardware-software boundary. You create, optimise and maintain data storages and platforms so that our users can access high quality and reliable data assets and solutions. You play a key role in transforming epics into delivering new...
-
Data Transformation Engineer
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India N Consulting Ltd Full timeData Transformation EngineerWe are seeking a highly skilled Data Transformation Engineer to join our team at N Consulting Ltd.About the Role:As a Data Transformation Engineer, you will be responsible for analyzing and modeling complex data structures and requirements.You will work closely with stakeholders to identify requirements and develop suitable...
-
Software Engineer
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India N Consulting Ltd Full timeWe are looking for a talented Software Engineer to join our team and contribute to the development of our backend systems. As a Backend Developer, you will be responsible for designing, implementing, and maintaining our server-side infrastructure.The ideal candidate will have a strong understanding of software development principles, including...
-
Data Engineering Senior Specialist
6 days ago
Pimpri-Chinchwad, Maharashtra, India Telstra Full timeRole SummaryWe are seeking a highly skilled Data Engineer with expertise in Spark, Python, Scala to join our team. As a Senior Data Engineer, you will play a key role in transforming epics into delivering new capability/product features for our customers.
-
Cybersecurity Threat Hunter
2 weeks ago
Pimpri-Chinchwad, Maharashtra, India Sonoco Full timeSonoco is seeking a highly skilled Cybersecurity Threat Hunter to join our team. This is a progression position, Level I-III, based on experience.Job DescriptionThe Cybersecurity Threat Hunter will be responsible for planning, conducting, and documenting iterative, hypothesis-based threat hunts to identify potential threats lurking within the IT...
-
Data Transformation Specialist
1 month ago
Pimpri-Chinchwad, Maharashtra, India N Consulting Ltd Full timeData Transformation SpecialistN Consulting Ltd is seeking a skilled Data Transformation Specialist to join our team. As a key member of our data engineering team, you will be responsible for analyzing and modeling complex data structures and requirements.Key Responsibilities:Analyze and model complex data structures and requirementsDevelop and implement data...
-
Senior Data Transformation Specialist
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India N Consulting Ltd Full timeData Transformation Engineer RoleN Consulting Ltd is seeking a highly skilled Technical Data Analyst to join our team in the position of Data Transformation Engineer. The ideal candidate will have a strong background in IT, particularly in technical analysis, data modeling, and software engineering.Key Responsibilities:Analyze and model complex data...
-
Python Software Engineer
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India N Consulting Ltd Full timeJob Title: Python Software EngineerN Consulting Ltd is looking for a skilled Python Software Engineer to join our team.Key Responsibilities:Develop and maintain Python applications to support various services.Perform performance tuning to optimize speed and efficiency.Implement cost optimization strategies for cloud-based services.Collaborate with DevOps...
-
Senior Data Transformation Specialist
7 days ago
Pimpri-Chinchwad, Maharashtra, India N Consulting Ltd Full timeN Consulting Ltd is seeking a highly skilled Senior Data Transformation Specialist to join our team.Estimated salary: $120,000 - $150,000 per year.Job DescriptionWe are looking for a talented professional with 7+ years of experience in IT, specifically in technical analysis, data modeling, or similar roles. The ideal candidate will have a solid understanding...
-
Senior Data Platform Specialist
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India Telstra Full timeJob Title: Data Engineering Senior SpecialistAt Telstra, we're looking for a highly skilled Data Engineer to join our team. As a Data Engineer, you'll play a key role in bridging the hardware-software boundary, creating, optimising, and maintaining data storages and platforms. Your expertise in designing, developing, and maintaining data pipelines using...
-
Senior Data Integration Specialist
6 days ago
Pimpri-Chinchwad, Maharashtra, India N Consulting Ltd Full timeAt N Consulting Ltd, we are seeking a highly skilled Senior Data Integration Specialist to join our team.The ideal candidate will have at least 7 years of experience in IT, with a background in technical analysis, data modeling, or a similar field. A strong understanding of fundamental coding concepts and the ability to analyze and model complex data...
-
Security Threat Hunter
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India Sonoco Full timeJob SummaryThe Sonoco team is seeking a highly skilled Security Threat Hunter to join our cybersecurity efforts. As a key member of our team, you will be responsible for identifying and mitigating potential security threats to our systems and infrastructure.Key Responsibilities:Plan and execute threat hunts to identify potential security risks.Analyze and...
-
Senior Financial Messaging Engineer
2 weeks ago
Pimpri-Chinchwad, Maharashtra, India N Consulting Ltd Full timeN Consulting LtdWe are seeking an experienced Senior Financial Messaging Engineer to join our team. This role will involve taking on responsibility for business requests from analysis to implementation in financial messaging.The ideal candidate will have a strong understanding of XML data and schemas (XSD) as well as experience with software engineering...
-
Senior Software Developer
7 days ago
Pimpri-Chinchwad, Maharashtra, India Telstra Full timeOverviewTelstra is a well-established Australian telecommunications and technology company with a strong presence in over 20 countries. Our mission is to build a connected future where everyone can thrive.SalaryThe estimated annual salary for this role is around $120,000 AUD, depending on experience and location.Job DescriptionWe are seeking an experienced...
-
Software Developer
1 month ago
Pimpri-Chinchwad, Maharashtra, India Telstra Full timeAbout the RoleWe are seeking a talented Software Developer to join our team at Telstra. As a Software Developer, you will be responsible for designing, building, and delivering innovative software products and solutions that delight our customers.Key ResponsibilitiesCollaborate with product owners, designers, and stakeholders to understand project...
-
Design Engineer
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India Executive81 HR Solutions Pvt. Ltd. Full timeAbout the RoleWe are searching for a skilled Design Engineer (Mechanical) to join our Design and Engineering Department at Executive81 HR Solutions Pvt. Ltd.This is a full-time opportunity for an experienced mechanical design engineer to contribute to the creation of innovative products.As a mechanical design engineer, you will be responsible for designing...
-
IT Operations Manager
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India Cts Express Logistics Pvt Full timeRole Overview: We are seeking a highly skilled IT Operations Manager to join our team at Cts Express Logistics Pvt. This role is responsible for maintaining the company's IT infrastructure, ensuring seamless operations, and driving business growth through innovative IT solutions. Key Responsibilities:• Analyze department needs and identify opportunities...
-
Product Design Engineer
6 days ago
Pimpri-Chinchwad, Maharashtra, India Executive81 HR Solutions Pvt. Ltd. Full timeAt Executive81 HR Solutions Pvt. Ltd., we are seeking a skilled Product Design Engineer - Mechanical to join our innovative team.We offer a competitive salary of ₹25,000 per month, based on experience.Key Responsibilities:Design new mechanical products and processes, leveraging your expertise in CAD software.Analyze prototype data, identify areas for...
-
Software Development Professional
2 weeks ago
Pimpri-Chinchwad, Maharashtra, India Telstra Full timeWe are seeking a skilled Software Development Professional to join our team at Telstra.Job OverviewTelstra is a well-established telecommunications and technology company with a strong presence in over 20 countries. We are committed to building a connected future for everyone, leveraging innovative technologies and solutions that transform the way we...