GCP Data Engineer
5 hours ago
We are seeking an experienced GCP Data Engineer to join our dynamic data engineering team. As a Data Engineer, you will design, develop, and maintain data pipelines on Google Cloud Platform (GCP) to support large-scale data analytics and machine learning initiatives. The ideal candidate will have a strong background in working with BigQuery, Python, and Databricks.
Key Responsibilities:- Design and Build Data Pipelines: Develop scalable and efficient data pipelines using GCP services such as BigQuery, Cloud Dataflow, Cloud Storage, and Pub/Sub.
- BigQuery Optimization: Optimize queries, build materialized views, and ensure the efficiency and performance of large datasets in BigQuery.
- Data Modeling and ETL: Create and maintain ETL (Extract, Transform, Load) processes, transforming raw data into structured, easily accessible formats for analytics.
- Automation and Scripting: Write Python scripts to automate data workflows, integrate data sources, and manage metadata for data governance.
- Databricks Integration: Work with Databricks for data processing, including the use of Apache Spark for batch and stream processing.
- Collaboration: Collaborate with Data Scientists, Analysts, and other stakeholders to understand data requirements and provide effective solutions.
- Data Quality and Governance: Ensure data integrity, quality, and security by following best practices for data governance, monitoring, and logging.
- Experience with GCP Services: Hands-on experience with Google Cloud Platform tools, especially BigQuery, Dataflow, Pub/Sub, Cloud Storage, Cloud Functions, and Cloud Composer.
- Proficiency in Python: Strong Python programming skills, especially for data engineering, data transformation, automation, and working with APIs.
- Databricks/Apache Spark: Experience working with Databricks or Apache Spark for distributed data processing and data pipeline orchestration.
- SQL Expertise: Advanced knowledge of SQL, specifically with BigQuery for creating complex queries and optimizing large-scale datasets.
- Data Modeling: Experience with data modeling and designing data architectures that support high-performance analytics and reporting.
- ETL Tools: Experience with ETL tools and frameworks (e.g., Apache Airflow, Dataflow, or dbt).
- Version Control: Experience with Git or similar version control systems.
- Cloud Data Architecture: Familiarity with best practices for cloud data architecture, including data lakes, data warehouses, and cloud-native technologies.
- Agile Methodology: Experience working in Agile development environments, collaborating in cross-functional teams.
- Data Engineering Frameworks: Familiarity with frameworks such as Apache Kafka, Apache Beam, or Flinkfor real-time data processing.
- Machine Learning Integration: Basic understanding of integrating data pipelines with machine learning models and platforms.
- Security & Compliance: Knowledge of GCP security best practices (e.g., IAM, data encryption, auditing) and compliance standards (e.g., GDPR, HIPAA).
- CI/CD Pipelines: Familiarity with setting up CI/CD pipelines for automated testing and deployment of data pipelines.
- Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field.
- 5+ years of experience as a Data Engineer, with a focus on GCP, data pipelines, and analytics.
-
GCP DevOps Engineer
3 months ago
palakkad, India Zemoso Technologies Full timeWhat You will doWe're looking for bright DevOps engineers that are experienced individual contributors and dynamic teamleaders, to join our dedicated DevOps Practice Group. As a DevOps practitioner at Zemoso you’ll useyour expertise to provide DevSecOps services to our customers and to our inhouse Dev teams.Basic Qualifications● Bachelor's degree in...
-
Data Engineer
4 months ago
Palakkad, India GeoTech Full timeAbout Us: We are an innovative technology company focused on delivering data-driven solutions that power our business operations and drive growth. We are looking for a skilled Data Engineer with deep expertise in Google Cloud solutions to join our team. This role is integral to building and maintaining our data infrastructure, ensuring scalability, security,...
-
Data Scientist
4 days ago
Palakkad, India Moative Full timeEnergy & Utilities Data ScientistAbout MoativeMoative, an Applied AI Services company, designs AI roadmaps, builds co-pilots and predictive AI solutions for companies in energy, utilities, packaging, commerce, and other primary industries. Through Moative Labs, we aspire to build micro-products and launch AI startups in vertical markets.Our Past: We have...
-
Data Engineer
4 weeks ago
Palakkad, India Truelancer.com Full timeJob Title: Data Engineer - DBT (Data Build Tool)Experience: 5-8 yrsMust-Have Skills: Airflow, S3, Snowflake, SQL/No SQL, Talend, AWSEssential Skills, Education and Experience:Should have a B.E. / B.Tech. / MCA or equivalent degree along with 4-7 years of experience in Data EngineeringStrong experience in DBT concepts like Model building and configurations,...
-
Data Governance Engineer
4 days ago
Palakkad, India Mastek Full timeJob SummaryWe are seeking a Data Governance Engineer to manage and enforce governance policies across Mashreq’s hybrid data ecosystem. This role is pivotal in ensuring data quality, compliance, and security while supporting data platform operations as the first line of support. The ideal candidate should have expertise in Informatica EDC, Axon, and Azure...
-
Senior Data Engineering Expert in Operations
5 days ago
Palakkad, Kerala, India ANSR Full timeWe are seeking a highly experienced Senior Data Engineering Expert to join our talented team at ANSR in Bangalore. This strategic role will lead our data engineering efforts, designing and implementing complex data solutions using Python, Spark, SQL, Snowflake, dbt, machine learning, and other relevant technologies.Key ResponsibilitiesLead the design,...
-
DevOps/SRE Engineer
3 days ago
Palakkad, Kerala, India BayOne Solutions Full timeSeeking an experienced DevOps/SRE Engineer to join our team at BayOne Solutions in Gurugram/Bengaluru. This role involves migrating teams to new deployment settings, updating Python scripts for deployment artifacts and pipelines, and maintaining Kubernetes operators.Key ResponsibilitiesMigrate teams from current deployment settings to new onesUpdate and...
-
Atrium | Data Engineering Lead | palakkad
3 weeks ago
palakkad, India Atrium Full timeUnlock yourself. Take your career to the next level.At Atrium, we live and deliver at the intersection of industry strategy, intelligent platforms, and data science — empowering our customers to maximize the power of their data to solve their most complex challenges. We have a unique understanding of the role data plays in the world today and serve as...
-
Senior Operational Engineer
4 weeks ago
Palakkad, Kerala, India CSG Full timeJob Title: Senior Operational EngineerAbout the Role:The ideal candidate will excel in a hybrid work environment, with the ability to choose their work style. This role requires a grower who can learn new skills and adapt quickly to enhance results. The individualist will succeed by leading their own projects and producing impactful outcomes. A time-manager...
-
Back End Engineer
2 weeks ago
Palakkad, India Ampera Full timeAmpera DEIAs an expert in DEI strategy and implementation leadership, Ampera brings thought leadership to diverse areas through customized, end-to-end services, including DEI Data-Driven Solutions and Diversity Scorecards, among others. Ampera provides organizations with comprehensive solutions to achieve their DEI goals, offering access to a diverse talent...
-
Data Annotator
4 days ago
Palakkad, India Jupiter AI Labs ✔ Full timePosition : Data Annotation fresherWorking Hours : Full Time (Slot : 1 PM to 9 PM)Salary : Up to 15kExperience : Fresher and Non TechNote : Only boys Preferred for this role and 1pm to 9pm slotWalking Venue : SECOND FLOOR, PLOT NO. A-61, B/4, Spring Meadows Business Park, Sector 63, Noida, Gautam buddha Nagar, Uttar Pradesh, 201301 Time : 10 am to 5pmNote :...
-
palakkad, India ClearML Full timeInformation Technology Manager, AI ComputingCompany DescriptionClearML is a unified, open source platform for continuous AI/ML, trusted by forward-thinking Data Scientists, ML Engineers, DevOps, and decision makers at leading Fortune 500, enterprises, academia, and innovative start-ups worldwide. We enable customers to achieve the fastest time to production,...
-
Data Analyst Executive
4 days ago
Palakkad, India Avow Solutions Inc Full timePosition OverviewWe are seeking a detail-oriented and analytical Data Analyst to join our team. The successful candidate will collect, process, and analyze data to provide actionable insights that drive business decisions. This role requires proficiency in data manipulation, statistical analysis, and data visualization.Key Responsibilities1.Data Collection...
-
Databricks Architect
1 month ago
Palakkad, India HS Solutions Inc Full timeAbout the JobWe are having an Urgent opening for one of our MNC client for PAN India Locations.Position Title: Databricks ArchitectNotice Period: Immediate to 30 days maximumLocation: PAN IndiaExperience: 10-15 YrsIn this role, the Databricks Architect is responsible for providing technical direction and lead agroup of one or more developer to address a...
-
Infoya | Sr. DevOps Engineer | palakkad
3 weeks ago
palakkad, India Infoya Full timeCompany DescriptionInfoya is a global IT consulting and services organization. We collaborate with visionary leaders to achieve outsized business impact by providing innovative solutions and exploring cutting-edge technologies. We work hand-in-hand with our clients to facilitate transformative change, guiding them through strategic shifts in technology,...
-
Data Science Team Lead
5 days ago
Palakkad, Kerala, India ZeroNoticePeriod Full timeAbout UsAt ZeroNoticePeriod, we're committed to innovation and excellence in our field.Job SummaryWe're seeking an experienced Data Science Team Lead to join our team. As a key member of our organization, you will be responsible for leading our data science efforts and driving business growth through data-driven insights.Key ResponsibilitiesLead the...
-
Senior Data Science Professional
5 days ago
Palakkad, Kerala, India ZeroNoticePeriod Full timeWe are seeking a highly skilled Senior Data Science Professional to join our team at ZeroNoticePeriod.As a key member of our data science team, you will be responsible for driving business growth through data-driven decision making.The ideal candidate will have a strong background in statistics, economics, or a related field, with a minimum of 5 years of...
-
Full Stack Engineer
4 days ago
Palakkad, India BeGig Full timeJob Description: Full Stack DeveloperNoida (In-office role)Budget-5-8 LPACompany Description:Client is a biotech company dedicated to solving diagnostic challenges anddeveloping therapeutic solutions for rare diseases through the power of genomics andartificial intelligence. Based in Noida, India, our interdisciplinary team comprisesscientists,...
-
Commissioning Engineer
1 month ago
Palakkad, India Commtech Asia Full timeJOB DUTIESPerform task as directed by the Project LeaderManage compilation of technical reports and submission of status report, including progress updateTechnically responsible for delivery of service in accordance with Commtech / Client expectationsReview of multi-discipline Testing and Commissioning documents for compliance with specification and...
-
YourDOST | Software Development Engineer II
3 days ago
Palakkad, India YourDOST Full timeWho we are:Founded in 2014 by Richa Singh and Puneet Manuja, YourDOST is one of India’s first and leading holistic emotional wellness organisations. We help corporates, colleges build resilient communities.We are enabling leading corporates like TCS, CK Birla Group, UpGrad, Darwinbox, Tata Consumer Products and educational institutions like IIT Madras, IIT...