Cloud Data Engineer Position with PySpark Experience
1 day ago
Mastech InfoTrellis, a leading provider of digital transformation solutions, is seeking an experienced Cloud Data Engineer to join our team.
As a Cloud Data Engineer at Mastech InfoTrellis, you will play a key role in designing, implementing, and maintaining scalable data pipelines on Google Cloud Platform (GCP) using PySpark. Your primary responsibility will be to develop and optimize high-quality data pipelines that ensure data accuracy, consistency, and reliability across various applications.
We are looking for a skilled Data Engineer who can work closely with cross-functional teams to understand data requirements and translate them into technical solutions. The ideal candidate should have hands-on experience with GCP services, particularly BigQuery, Dataflow, Pub/Sub, and Cloud Storage. In addition, expertise in designing, implementing, and optimizing data pipelines using Databricks in GCP is essential.
Key Responsibilities:
- Design, develop, and maintain data pipelines on GCP using PySpark for data processing, transformation, and analysis.
- Work closely with cross-functional teams to understand data requirements and translate them into technical solutions.
- Optimize and tune data pipelines for performance, scalability, and reliability.
- Implement best practices for data governance, security, and compliance.
- Collaborate with Data Scientists and Analysts to support their data needs and ensure data availability and quality.
- Troubleshoot and debug data pipeline issues in production environments.
- Stay up-to-date with the latest trends and technologies in data engineering and cloud computing.
- Contribute to the continuous improvement of our data engineering practices and processes.
Requirements:
- Proven experience as a Data Engineer familiar with Databricks and GCP services.
- Strong proficiency in Google Cloud Platform (GCP) services, particularly BigQuery, Dataflow, Pub/Sub, and Cloud Storage.
- Strong experience designing, implementing, and optimizing data pipelines using Databricks in GCP.
- Hands-on experience with PySpark for data processing and analysis.
- Solid understanding of distributed computing principles and experience with large-scale data processing frameworks.
- Experience with SQL and NoSQL databases.
- Familiarity with data modelling, ETL, and data warehousing concepts.
- Excellent problem-solving and communication skills.
- Ability to work effectively in a fast-paced, dynamic environment.
Preferred Skills & Qualifications:
- Bachelor's or Master's degree in Computer Science & Engineering, or a related field.
- Google Cloud Platform (GCP) certification.
- Experience with other big data technologies such as Hadoop, Spark, or Apache Beam.
- Knowledge of machine learning concepts and tools.
- Experience with containerization technologies like Docker and Kubernetes.
The salary range for this position is approximately $120,000 - $180,000 per annum, depending on qualifications and experience.
-
Cloud Data Engineer Position
3 weeks ago
Chennai, Tamil Nadu, India Mastech InfoTrellis Full timeAbout the Role:Mastech InfoTrellis is seeking a highly skilled Cloud Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, implementing, and maintaining scalable data pipelines on Google Cloud Platform (GCP) using PySpark for data processing, transformation, and analysis.Key...
-
Chennai, Tamil Nadu, India Mastech InfoTrellis Full timeMastech InfoTrellis seeks a seasoned Data Engineer to design and implement large-scale data pipelines on Google Cloud Platform (GCP).Estimated Salary: $120,000 - $180,000 per year.About the Role:We are looking for an experienced professional with expertise in GCP services, particularly BigQuery, Dataflow, Pub/Sub, and Cloud Storage. The ideal candidate will...
-
Data Engineer Lead Position in Chennai
1 week ago
Chennai, Tamil Nadu, India Encora Inc. Full timeAre you a skilled data professional looking for a challenging role?We have an exciting opportunity for a Data Engineer Lead to join our team at Encora Inc. in Chennai.This is a full-time position that requires 8-10 years of experience in data engineering and a strong background in big data/cloud technologies like Pyspark, Kafka, etc.The ideal candidate will...
-
Pyspark Developer
1 month ago
Chennai, Tamil Nadu, India Krishpar Technologies Pvt. Ltd Full timeAbout the RoleWe are seeking an experienced Data Engineer with Pyspark expertise to join our team at Krishpar Technologies Pvt. Ltd.Key ResponsibilitiesDevelop Big Data applications using Pyspark on Hadoop, Hive, and/or Kafka, HBase, and MongoDB.Build Machine Learning models using Pyspark.Deploy applications on Cloud platforms.RequirementsAt least 4-7 years...
-
Cloud Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India Mastech InfoTrellis Full timeAbout the RoleMastech InfoTrellis is seeking a talented Cloud Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing and implementing scalable data pipelines on Google Cloud Platform (GCP) using PySpark for processing and analysis.ResponsibilitiesDesign, develop, and maintain data pipelines on GCP...
-
Senior Data Analyst Position
1 day ago
Chennai, Tamil Nadu, India Revature Full timeAbout the RoleWe are seeking a highly skilled Data Engineer to join our team at Revature. This is an exciting opportunity for an individual with expertise in data integration, transformation, and analysis.This role involves designing, developing, and optimizing ETL pipelines using PySpark and Azure Databricks. You will work closely with cross-functional...
-
Senior Data Engineer
2 days ago
Chennai, Tamil Nadu, India Mastech InfoTrellis Full timeAbout the RoleMastech InfoTrellis is seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, implementing, and maintaining scalable data pipelines on Google Cloud Platform (GCP) using PySpark.Responsibilities:- Design, develop, and maintain data pipelines on GCP using PySpark for data...
-
Chennai, Tamil Nadu, India Mastech InfoTrellis Full timeWe are seeking a talented Cloud Engineer to join our team at Mastech InfoTrellis. As a Cloud Engineer, you will be responsible for designing, implementing, and maintaining scalable data pipelines and solutions on Google Cloud Platform (GCP) using PySpark for processing and analysis.Job DescriptionThe successful candidate will have expertise in GCP services,...
-
Chennai, Tamil Nadu, India Mastech InfoTrellis Full timeJob Summary: We are seeking a talented and motivated Data Engineer with expertise in Google Cloud Platform (GCP) to join our team at Mastech InfoTrellis.About the Role:This is a challenging and exciting opportunity to design, implement, and maintain scalable data pipelines and solutions on GCP using PySpark for processing and analysis. As a Data Engineer,...
-
Chennai, Tamil Nadu, India Colan Infotech Private Limited Full timeWe are Colan Infotech Private Limited, a renowned company in the field of data engineering.Our ideal candidate for this role will have at least 8 years of experience working with Azure Data Factory, Databricks, and PySpark/Scala Spark.The salary range for this position is around ₹12.5 lakhs per annum, considering the cost of living in Chennai and the level...
-
Cloud Infrastructure Data Engineer Position
2 weeks ago
Chennai, Tamil Nadu, India Element Infomatics (India) Pvt. Ltd. Full timeCompany OverviewElement Infomatics (India) Pvt. Ltd. is a technology-driven company based in Bangalore, committed to delivering innovative IT services using flexible engagement models. We collaborate with global industry-leading technologies to provide high-value solutions across various sectors.About the JobThis is a full-time hybrid role for a Cloud Data...
-
AWS Data Engineering Lead Position
2 weeks ago
Chennai, Tamil Nadu, India Tata Consultancy Services Full timeAbout UsTata Consultancy Services is a leading IT services company that helps organizations transform and grow their businesses. We are a global leader in consulting and technology, combining our deep industry knowledge with innovative solutions to deliver results for our clients.Job OverviewWe are currently hiring for an AWS Data Engineer Lead position....
-
Cloud Data Engineer
2 weeks ago
Chennai, Tamil Nadu, India Delta Systech India Pvt Ltd. Full timeDelta Systech India Pvt Ltd. is seeking a highly skilled Cloud Data Engineer to join our team in Chennai, working 3 days a week in a hybrid environment.Job DescriptionWe are looking for an experienced Big Data Developer to design, implement, and maintain large-scale data warehousing solutions using Azure Synapse and Data Bricks. The ideal candidate will have...
-
Cloud Data Warehouse Developer
1 month ago
Chennai, Tamil Nadu, India Delta Systech India Pvt Ltd. Full timeData Engineer - Azure SynapseAbout the RoleDelta Systech India Pvt Ltd. is seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data warehouses using Azure Synapse and Pyspark.Key ResponsibilitiesDevelop and implement ETL/Pipeline Development using Azure...
-
Cloud Data Engineer
15 hours ago
Chennai, Tamil Nadu, India Mastech InfoTrellis Full timeWe are seeking a talented and experienced Data Engineer with expertise in Google Cloud Platform (GCP) to join Mastech InfoTrellis as a key member of our team.About the RoleThe successful candidate will be responsible for designing, developing, and maintaining scalable data pipelines and solutions on GCP using PySpark for processing and analysis. As a Data...
-
Chennai, Tamil Nadu, India Unison Consulting Pte Ltd Full timeAs a key player in our team, you will be responsible for the successful migration of data from SAS to PySpark.We are seeking a highly skilled Data Integration Specialist to join our Unison Consulting Pte Ltd team. This is an exciting opportunity to work on a large-scale data migration project and take advantage of your expertise in data integration, ETL, and...
-
Data Engineer
2 weeks ago
Chennai, Tamil Nadu, India Colan Infotech Pvt Ltd Full time**Company Overview:**Colan Infotech Pvt Ltd is a leading technology company that specializes in providing innovative solutions to its clients. Our team of experts works collaboratively to deliver high-quality projects that meet the evolving needs of our customers.**Estimated Salary:** ₹1,200,000 - ₹1,800,000 per annum**Job Description:**We are seeking an...
-
Data Engineer Lead
1 week ago
Chennai, Tamil Nadu, India Genpact Full timeGenpact, a global professional services and solutions firm, seeks a Data Engineer Lead with expertise in cloud architecture to drive business outcomes. As a seasoned leader, you will spearhead the development of data pipelines, ensuring seamless integration with client systems. With a strong background in AWS technologies, including Glue, Athena,...
-
Cloud Data Engineer Lead
3 weeks ago
Chennai, Tamil Nadu, India Fulcrum Digital Full timeCompany OverviewFulcrum Digital is a forward-thinking digital acceleration company that provides comprehensive digital transformation and technology services. Our expertise spans multiple industries, including banking & financial services, insurance, retail, higher education, food, healthcare, and manufacturing.About the RoleWe seek an experienced...
-
Chennai, Tamil Nadu, India T D Newton Full timeSalary: ₹24LPA - ₹40LPAT D Newton is Hiring:Job DescriptionWe are seeking an experienced Data Engineer to join our team in Gurgaon or Chennai. The ideal candidate will have a strong background in designing and developing robust data pipelines for structured and unstructured data, utilizing cloud-based platforms like Redshift, Snowflake, or Databricks...