Senior Big Data Engineer
2 weeks ago
Position - Senior Big Data Engineer (GCP)
Experience - 5 to 8 Yrs
Location - Remote
Shift - EST Shift 5:30 PM to 2:30 AM
We are seeking a highly skilled Senior Big Data Engineer with extensive experience in Big Data technologies, including Hadoop, Hive, PySpark, and expertise in Google Cloud Platform (GCP) and BigQuery. The successful candidate will be responsible for designing, building, and maintaining robust data pipelines that support large-scale data processing and analytics. This role requires excellent data analysis skills and the ability to architect solutions for cloud-based environments, with a focus on extracting, processing, and analyzing data from diverse sources.
Key Responsibilities :
- Design, develop, and optimize Big Data pipelines and workflows using Hadoop, Hive, PySpark, and Google BigQuery.
- Architect data solutions to facilitate smooth data migration and processing between on-premises systems and Google Cloud Platform (GCP).
- Create and manage Unix Shell Scripts for process automation and data handling tasks.
- Perform in-depth data analysis and work comfortably with large datasets to derive actionable insights.
- Collaborate effectively with cross-functional teams to understand data requirements and translate them into scalable data solutions.
- Ensure data quality, integrity, and security across all stages of the data lifecycle.
- Stay updated with the latest industry trends and emerging technologies to continuously improve data solutions.
Mandatory Qualifications :
- Minimum of 8 years of industry experience working with Big Data technologies.
- Proficiency in Hadoop, Hive, and PySpark for data processing.
- Strong experience with Unix Shell Scripting.
- Expertise in SQL and extracting data from multiple sources.
- Demonstrated experience in data analysis, specifically querying and analyzing large datasets on Hadoop HDFS using Hive and Spark.
- Proven experience in architecting Big Data pipelines and data solutions.
- Proficiency in Google BigQuery and migrating data pipelines to GCP.
- Exceptional communication skills for clear requirement gathering and collaboration.
Preferred Qualifications :
- Google Cloud Platform (GCP) certification.
- Professional experience with cloud hosting platforms, with a preference for GCP.
-
Big Data Engineer
1 month ago
Chennai/Bangalore/Hyderabad/Mumbai/Pune/Delhi/Coimbatore, IN Flipped.ai Full timeJob Description : Big Data Engineer - Scala, SparkAbout the Role :We are seeking a highly skilled Big Data Engineer to join our team and help us leverage the power of big data to drive innovation and business value. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining scalable and efficient big data solutions. You will...
-
Big Data Engineer
1 month ago
Hyderabad/Pune/Delhi NCR, IN Ace Recruit Full timeJob Description :As a Big Data Engineer (Azure), you will build and learn about a variety of analytics solutions & platforms, data lakes, modern data platforms, data fabric solutions, etc. using different Open Source, Big Data, and Cloud technologies on Microsoft Azure. On a typical day, you might :- Design and build scalable & metadata-driven data ingestion...
-
Data Engineer
1 month ago
Bangalore/Pune/Trivandrum/Thiruvananthapuram, IN Rao Career Solutions Full timeJob Summary :- We are looking for a skilled Data Engineer with hands-on experience in PySpark and Big Data platforms to join our growing team. The ideal candidate will have strong expertise in building data pipelines, working with big data technologies like Apache Hive and Parquet, and handling diverse data use cases. You will work closely with data...
-
Big Data Engineer
1 month ago
Chennai/Hyderabad/Bangalore/Coimbatore/Mumbai/Delhi/Pune, IN GAIUS HYPERLOCAL PRIVATE LIMITED Full timeJob Description : Big Data Engineer - Scala, SparkYears of exp: 6 to 13 yearsJob location: Chennai / Hyderabad / Bangalore / Pune / Mumbai / Delhi / CoimbatoreAbout the Role :We are seeking a highly skilled Big Data Engineer to join our team and help us leverage the power of big data to drive innovation and business value. As a Big Data Engineer, you will be...
-
Big Data Engineer
1 month ago
Bangalore/Pune/Mumbai/Chennai, IN a client of change leaders Full timeJob Description : - 5+ years of experience developing scalable Big Data applications or solutions on distributed platforms- Design and implement data pipelines using Snowflake, PostgreSQL, and Python.- Experience in AWS/ Azure/ GCP and good to have other cloud platform tools- Experience working with Big Data platforms, including Hadoop- Hive, Data Bricks...
-
Big Data Engineer
1 month ago
Chennai/Pune, IN Pan Asia Man power Consultants Recruiting agency Full timeJob Title : Big Data EngineerExperience Range : 6+ YearsLocation : Chennai/Pune (Hybrid - 2 Days Work From Office in a Week)Job Description :We are seeking a skilled Big Data Engineer to join our Agile team and contribute to building backend data applications and services that support our analytics platforms. This role requires a hands-on developer with...
-
Senior Data Integration Engineer
1 month ago
Bangalore, Pune ,Mumbai,Noida,Chennai,Hyderabad, IN RAPINNO TECH SOLUTIONS PRIVATE LIMITED Full timeJob Title : Senior Data Integration Engineer (Talend + Azure + Databricks)Location : Bangalore, Pune, Mumbai, Noida, Chennai, HyderabadWorking Hours : 12 PM IST to 9 PM ISTExperience Required : 6 to 8 YearsNotice Period : Immediate to 3 Weeks MaxJob Description :We are seeking a highly skilled Senior Data Integration Engineer with expertise in Talend and...
-
Lead Databricks Engineer
4 weeks ago
Udaipur/Jaipur/Bangalore/Kolkata, IN Kadel Labs Private Limited Full timeKadel Labs is a first-of-its-kind Venture Studio in India. It is a global software technology system incubator with synergetic blend of IT Services and SaaS products under its portfolio. Since 2017, KL has grown organically (180+ employees and expanding) and served 200+ Customers. At the core, KL is a people's company. We serve clients by practicing an...
-
Big Data Engineer
1 month ago
Chennai/Noida/Gurgaon/Gurugram/Mumbai, IN ANRI Solutions HR Services Full timeRole : Big Data Engineer Looking for an experienced Spark and Scala developer able to create an end-to-end data pipeline.Associate should be self-driven, can work with minimal guidance and guide the team technically. Job Details : 1. Expertise in Spark framework with scala language and Pyspark 2. Expertise in ETL and Data Warehousing and cloud concepts 3....
-
Big Data Engineer
5 months ago
Goa/Mumbai/Jammu & Kashmir/Jammu/Srinagar/Pondicherry/Jaipur/Lucknow/Varanasi/Banaras/Patna/Ranchi, IN ATech Full timeDesignation: BIG DATA ENGINEERJob Description:Your Role and Responsibilities:- Understand a data warehousing solution and able to work independently in such an environment- Responsible in Project development and delivery experience of a few good size projects- Design, build, optimize and support new and existing data models and ETL processes based on our...
-
Big Data Administrator
4 weeks ago
Bangalore/Gurgaon/Gurugram/Nagpur, IN Intuitive Apps Full timeAs a Big Data Admin, you will play a crucial role in ensuring the smooth operation and optimization of our Hadoop and Spark infrastructure. Your responsibilities will include :Hadoop Installation and Deployment :- Install Hadoop in Linux environments.- Deploy and maintain Hadoop clusters.- Monitor cluster health to ensure continuous uptime.- Analyze storage...
-
Big Data Developer
1 month ago
Pune/Gurgaon/Gurugram, IN Talent Toppers LLP Full timeResponsibilities :- Undertake complete ownership in accomplishing activities and assigned responsibilities across all phases of project lifecycle to solve business problems across one or more client engagements;- Apply appropriate development methodologies (e.g.: agile, waterfall) and best practices (e.g.: mid-development client reviews, embedded QA...
-
Big Data Team Lead
1 month ago
Bangalore/Chennai/Pune, IN Rigel networks Full timeJob Title : Senior Software Developer / Applications Development ManagerLocation : Pune, Chennai, Bangalore Shift Timings : 10 AM - 7 PMPosition Overview : We are seeking an experienced Senior Software Developer / Applications Development Manager with a strong background in software development and project management. The ideal candidate will possess...
-
Hadoop Administrator
4 months ago
Mumbai/Bangalore/Bhubaneshwar/Anywhere in India/Multiple Locations, IN Tehno Right Full timeJob Role : Hadoop Administrator (Role open for multiple locations) - WFH and WFOJob description :What is your Role ?- You will manage Hadoop clusters, data storage, server resources, and other virtual computing platforms. - You perform a variety of functions, including data migration, virtual machine set-up and training, troubleshooting end-user problems,...
-
Lead Azure Data Engineer
4 months ago
Navi Mumbai/Pune/Bangalore, IN Techno Wise Full timeJob Summary : We are seeking a highly skilled and motivated individual to join our team as a Senior Azure Data Engineer. The ideal candidate will have a strong background in Azure data technologies, a proven track record of ability to design and implement scalable and efficient data solutions.Responsibilities :- Design, develop, and deploy data pipelines and...
-
ZS - Engineering Lead - Big Data
1 month ago
Pune/Gurgaon/Gurugram/Bangalore, IN ZS Full timeComapany Description :ZS is a place where passion changes lives. As a management consulting and technology firm focused on transforming global healthcare and beyond, our most valuable asset is our people. Here you'll work side-by-side with a powerful collective of thinkers and experts shZS is a place where passion changes lives. As a management...
-
Data Engineer II
1 month ago
Bangalore/Pune/Mumbai/Coimbatore/Delhi NCR/Delhi/Navi Mumbai, IN Concept Management Full timeRole - Data Engineer 2Job Type - Full-time, Permanent Qualification - BE or BtechNumber Of Roles - 15Location - Bangalore, Pune, Mumbai, Delhi, Gurugram, CoimbatorePackage - 15 to 20 LPAExperience - 3.5 to 6 YearsShift - UK Shift15 to 30 Days JoinerSkills : Data Engineering, Big data, ETL, Python, PySpark and SparkSQ, SQL, JIRA, Git, and GitHub.Job Summery...
-
TVARIT - Data Engineer - ETL
1 month ago
Remote/Pune, IN TVARIT Full timeJD of Senior Data Engineer : TVARIT GmbH develops and delivers solutions in the field of artificial intelligence (AI) for the Manufacturing, automotive, and process industries. With its software products, TVARIT makes it possible for its customers to make intelligent and well-founded decisions, e.g., in forward-looking Maintenance, increasing the OEE and...
-
Senior Data Scientist
1 month ago
Bangalore/Pune, IN Elite recruitments Full timeAbout the Role :We are seeking a highly skilled and experienced Senior Data Science Specialist to join our team. This role requires an individual with 8-10 years of experience in data science, who can leverage their expertise to drive data-driven decision-making and provide actionable insights. The ideal candidate will have a strong background in statistical...
-
Hadoop Administrator
5 months ago
Jaipur/Ahmedabad/Anywhere in India/Multiple Locations, IN ATech Full timeJob Profile : Hadoop AdministratorWFH and WFO (Hiring Office Jaipur and Ahmedabad)Role: Big Data EngineerIndustry Type: IT Services & ConsultingJob description :- Good understanding of SDLC and agile methodologies- Installation and configuration of Hadoop clusters, including HDFS, MapReduce, Hive, Pig, HBase, and other related tools- Managing and monitoring...