Data Engineer for Cloud Infrastructure
1 week ago
Telstra is a leading telecommunications and technology company with a rich heritage built over 100 years. With operations in more than 20 countries, including India, we're combining innovation, automation, and technology to solve the world's biggest technological challenges.
Salary:$130,000 - $180,000 per annum
Job Description
We're seeking a highly skilled Data Engineer with expertise in Spark, Python, Scala to design, develop, and maintain data pipelines using Spark, Python, Scala, and related technologies on Hadoop/Clouderra Platform.
Key Responsibilities
- Design, develop, and maintain data pipelines using Spark, Python, Scala, and related technologies on Hadoop/Clouderra Platform
- Work with high volume data and ensure data quality and accuracy
- Implement data security and privacy best practices to protect sensitive data
- Develop and maintain documentation on data pipeline architecture, data models, and data workflows
- Monitor and troubleshoot data pipelines to ensure they are performing optimally
- Stay up-to-date with the latest developments in Azure, AWS, Spark, Python, Scala, and related technologies and apply them to solve business problems
Required Skills and Qualifications
- Hands-on experience in Spark Core, Spark SQL, SQL/Hive/Impala
- Exposure on Hadoop Ecosystem (HDP/Cloudera/MapR/EMR etc)
- Experience of working on File formats (Parquet/ORC/AVRO/Delta/Hudi etc.)
- Experience with high volume data processing and data streaming technologies
- Experience of using Orchestration tools like Control-m
- Strong experience in data modeling, schema design, and ETL development using SQL and related technologies
- Familiarity with data security and privacy best practices
- Good exposure on TDD
- Exposure on using CI tools like Git, Bitbucket, Github, Gitlab, Azure DevOps
- Exposure on using CD tools like Jenkins, Bamboo, Azure DevOps
- Cloud exposure (Hadoop)
- Experience and knowledgeable on the following: Azure data offerings - ADF, ADLS2, Azure Databricks, Azure Synapse, Eventhubs, CosmosDB etc, Presto/Athena
- Exposure of working on Power BI
- Prior experience in building or working in team building reusable frameworks
- Good understanding of Data Architecture and design principles. (Delta/Kappa/Lambda architecture)
- Exposure to Code Quality - Static and Dynamic code scans
- Good knowledge of NoSQL Databases/ HBase/ MongoDB / Cassandra / Cosmos DB
- Good knowledge of GraphDB(Neo4J)
- Experience with enterprise data management, Datawarehouse, data modelling, Business Intelligence, data integration
- Expertise in SQL and stored procedure
- Experience in designing solutions for multiple large data warehouses with a good understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS and/or knowledge on NoSQL platforms
- Experience in working on Azure SQL Data warehouse (Synapse) and Azure analysis service or Redshift or Synapse
-
Data Engineering Developer
1 month ago
Pimpri-Chinchwad, Maharashtra, India Telstra Full timeJob TitleData Engineering DeveloperAbout the RoleWe are seeking a highly skilled Data Engineer with expertise in Spark, Python, and Scala to join our team at Telstra. As a Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines using Spark, Python, Scala, and related technologies.Key ResponsibilitiesDesign, develop,...
-
Data Engineering Senior Specialist
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India Telstra Full timeJob TitleData Engineering Senior SpecialistJob SummaryAs a Data Platform Engineer Senior Specialist, you bridge the hardware-software boundary. You create, optimise and maintain data storages and platforms so that our users can access high quality and reliable data assets and solutions. You play a key role in transforming epics into delivering new...
-
Data Transformation Engineer
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India N Consulting Ltd Full timeData Transformation EngineerWe are seeking a highly skilled Data Transformation Engineer to join our team at N Consulting Ltd.About the Role:As a Data Transformation Engineer, you will be responsible for analyzing and modeling complex data structures and requirements.You will work closely with stakeholders to identify requirements and develop suitable...
-
Data Engineering Senior Specialist
2 weeks ago
Pimpri-Chinchwad, Maharashtra, India Telstra Full timeRole SummaryWe are seeking a highly skilled Data Engineer with expertise in Spark, Python, Scala to join our team. As a Senior Data Engineer, you will play a key role in transforming epics into delivering new capability/product features for our customers.
-
Cybersecurity Threat Hunter
3 weeks ago
Pimpri-Chinchwad, Maharashtra, India Sonoco Full timeSonoco is seeking a highly skilled Cybersecurity Threat Hunter to join our team. This is a progression position, Level I-III, based on experience.Job DescriptionThe Cybersecurity Threat Hunter will be responsible for planning, conducting, and documenting iterative, hypothesis-based threat hunts to identify potential threats lurking within the IT...
-
Senior Data Transformation Specialist
2 weeks ago
Pimpri-Chinchwad, Maharashtra, India N Consulting Ltd Full timeN Consulting Ltd is seeking a highly skilled Senior Data Transformation Specialist to join our team.Estimated salary: $120,000 - $150,000 per year.Job DescriptionWe are looking for a talented professional with 7+ years of experience in IT, specifically in technical analysis, data modeling, or similar roles. The ideal candidate will have a solid understanding...
-
Senior Data Integration Specialist
2 weeks ago
Pimpri-Chinchwad, Maharashtra, India N Consulting Ltd Full timeAt N Consulting Ltd, we are seeking a highly skilled Senior Data Integration Specialist to join our team.The ideal candidate will have at least 7 years of experience in IT, with a background in technical analysis, data modeling, or a similar field. A strong understanding of fundamental coding concepts and the ability to analyze and model complex data...
-
Security Threat Hunter
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India Sonoco Full timeJob SummaryThe Sonoco team is seeking a highly skilled Security Threat Hunter to join our cybersecurity efforts. As a key member of our team, you will be responsible for identifying and mitigating potential security threats to our systems and infrastructure.Key Responsibilities:Plan and execute threat hunts to identify potential security risks.Analyze and...
-
Senior Financial Messaging Engineer
3 weeks ago
Pimpri-Chinchwad, Maharashtra, India N Consulting Ltd Full timeN Consulting LtdWe are seeking an experienced Senior Financial Messaging Engineer to join our team. This role will involve taking on responsibility for business requests from analysis to implementation in financial messaging.The ideal candidate will have a strong understanding of XML data and schemas (XSD) as well as experience with software engineering...
-
Senior Software Developer
2 weeks ago
Pimpri-Chinchwad, Maharashtra, India Telstra Full timeOverviewTelstra is a well-established Australian telecommunications and technology company with a strong presence in over 20 countries. Our mission is to build a connected future where everyone can thrive.SalaryThe estimated annual salary for this role is around $120,000 AUD, depending on experience and location.Job DescriptionWe are seeking an experienced...
-
Design Engineer
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India Executive81 HR Solutions Pvt. Ltd. Full timeAbout the RoleWe are searching for a skilled Design Engineer (Mechanical) to join our Design and Engineering Department at Executive81 HR Solutions Pvt. Ltd.This is a full-time opportunity for an experienced mechanical design engineer to contribute to the creation of innovative products.As a mechanical design engineer, you will be responsible for designing...
-
Technical Director
3 days ago
Pimpri-Chinchwad, Maharashtra, India Tata Technologies Full timeWe are looking for a Technical Director at Tata Technologies.The selected consultant will be required to support in defining and finalizing SOWs with suppliers, preparing detailed implementable specifications for all use cases including functional, network, and cloud interface architecture definition. The consultant must also prepare specifications related...
-
IT Operations Manager
1 month ago
Pimpri-Chinchwad, Maharashtra, India Cts Express Logistics Pvt Full timeRole Overview: We are seeking a highly skilled IT Operations Manager to join our team at Cts Express Logistics Pvt. This role is responsible for maintaining the company's IT infrastructure, ensuring seamless operations, and driving business growth through innovative IT solutions. Key Responsibilities:• Analyze department needs and identify opportunities...
-
Product Design Engineer
2 weeks ago
Pimpri-Chinchwad, Maharashtra, India Executive81 HR Solutions Pvt. Ltd. Full timeAt Executive81 HR Solutions Pvt. Ltd., we are seeking a skilled Product Design Engineer - Mechanical to join our innovative team.We offer a competitive salary of ₹25,000 per month, based on experience.Key Responsibilities:Design new mechanical products and processes, leveraging your expertise in CAD software.Analyze prototype data, identify areas for...
-
Software Development Professional
3 weeks ago
Pimpri-Chinchwad, Maharashtra, India Telstra Full timeWe are seeking a skilled Software Development Professional to join our team at Telstra.Job OverviewTelstra is a well-established telecommunications and technology company with a strong presence in over 20 countries. We are committed to building a connected future for everyone, leveraging innovative technologies and solutions that transform the way we...
-
Technical Project Office Lead for Global IT
1 week ago
Pimpri-Chinchwad, Maharashtra, India MAHLE Full timeJob DescriptionWe are seeking a highly experienced Technical Project Office Lead to join our global IT team at MAHLE. This is an exceptional opportunity for a seasoned professional to take on the development of our new Project Management solution.About the Role:As Technical Project Office Lead, you will be responsible for developing and implementing the...
-
Digital Transformation Leader
7 days ago
Pimpri-Chinchwad, Maharashtra, India OptimHire Full timeJob Title: Digital Transformation LeaderAbout Us: At OptimHire, we empower businesses to thrive in an ever-changing digital landscape. Our team of experts is dedicated to delivering innovative solutions that drive growth and success.Job Description:We are seeking an experienced Digital Transformation Leader to join our team. As a key member of our...
-
Software Development Expert
11 hours ago
Pimpri-Chinchwad, Maharashtra, India TechVerito Full timeAbout TechVerito">TechVerito is a leading technology company that delivers innovative solutions to clients worldwide.">Job Summary">We are seeking an experienced Full Stack Engineer to join our team. The ideal candidate will have a strong background in developing web applications using Java and React, as well as experience with microservices-based...
-
Failure Analysis Specialist
1 day ago
Pimpri-Chinchwad, Maharashtra, India EMotorad Full timeAbout the Role:We are seeking a highly skilled Failure Analysis Specialist to join our team at EMotorad.As a key member of our quality control department, you will play a crucial role in ensuring the reliability and performance of our products.With a strong background in mechanical engineering, materials science, or a related field, you will investigate...
-
Digital Solutions Architect
4 weeks ago
Pimpri-Chinchwad, Maharashtra, India OptimHire Full timeJob Title: Principal Solutions ArchitectJob Description:We're seeking an experienced Principal Solutions Architect to lead our Digital Practice. As a key member of our team, you'll design and architect complex solutions that meet our clients' business objectives. You'll collaborate closely with clients, project managers, and cross-functional teams to drive...