Senior Big Data Engineer
2 days ago
Position - Senior Big Data Engineer (GCP)
Experience - 5 to 8 Yrs
Location - Remote
Shift - EST Shift 5:30 PM to 2:30 AM
We are seeking a highly skilled Senior Big Data Engineer with extensive experience in Big Data technologies, including Hadoop, Hive, PySpark, and expertise in Google Cloud Platform (GCP) and BigQuery. The successful candidate will be responsible for designing, building, and maintaining robust data pipelines that support large-scale data processing and analytics. This role requires excellent data analysis skills and the ability to architect solutions for cloud-based environments, with a focus on extracting, processing, and analyzing data from diverse sources.
Key Responsibilities :
- Design, develop, and optimize Big Data pipelines and workflows using Hadoop, Hive, PySpark, and Google BigQuery.
- Architect data solutions to facilitate smooth data migration and processing between on-premises systems and Google Cloud Platform (GCP).
- Create and manage Unix Shell Scripts for process automation and data handling tasks.
- Perform in-depth data analysis and work comfortably with large datasets to derive actionable insights.
- Collaborate effectively with cross-functional teams to understand data requirements and translate them into scalable data solutions.
- Ensure data quality, integrity, and security across all stages of the data lifecycle.
- Stay updated with the latest industry trends and emerging technologies to continuously improve data solutions.
Mandatory Qualifications :
- Minimum of 8 years of industry experience working with Big Data technologies.
- Proficiency in Hadoop, Hive, and PySpark for data processing.
- Strong experience with Unix Shell Scripting.
- Expertise in SQL and extracting data from multiple sources.
- Demonstrated experience in data analysis, specifically querying and analyzing large datasets on Hadoop HDFS using Hive and Spark.
- Proven experience in architecting Big Data pipelines and data solutions.
- Proficiency in Google BigQuery and migrating data pipelines to GCP.
- Exceptional communication skills for clear requirement gathering and collaboration.
Preferred Qualifications :
- Google Cloud Platform (GCP) certification.
- Professional experience with cloud hosting platforms, with a preference for GCP.
-
Senior Data Integration Engineer
2 weeks ago
Kolkata, India Senior Data Integration Engineer Full timeMust Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...
-
delhi, India Senior Data Integration Engineer Full timeMust Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...
-
Senior Data Integration Engineer
2 weeks ago
Delhi, India Senior Data Integration Engineer Full timeMust Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...
-
Senior Data Integration Engineer
2 weeks ago
Delhi, India Senior Data Integration Engineer Full timeMust Have Skills/Skill Requirement:- Design and architect integration solutions to connect various enterprise applications, systems, and databases.- Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.- Utilize Azure Integration Services such as Azure Logic...
-
Senior data integration engineer
2 weeks ago
Delhi, India Senior Data Integration Engineer Full timeMust Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...
-
Senior Big Data Developer
4 weeks ago
Pune, Maharashtra, India NTT DATA Full timeCompany OverviewNTT DATA is a global technology services company that helps clients innovate, optimize and transform for long term success. We have diverse experts in more than 50 countries and a robust partner ecosystem of established and start-up companies.Our Commitment to YouEstimated Salary: ₹2,200,000 - ₹3,500,000 per annum (dependent on...
-
Senior Data Engineer
6 months ago
Pune, India Data Axle Full timeJob Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...
-
Senior Data Engineer
4 weeks ago
Pune, Maharashtra, India TapTalent Full timeTapTalent is a leading player in the job market, and we're looking for a seasoned Data Engineer to join our team. As a senior member of our engineering team, you will play a crucial role in designing and developing our big data platform.We're looking for someone with at least 4 years of experience in working with Scala Spark. If you have expertise in data...
-
Senior Data Engineer
2 weeks ago
Delhi, Delhi, India PRONEXUS CONSULTING PRIVATE LIMITED Full time**Job Title:** Senior Data Engineer - Big Data Pipeline SpecialistAt Pronexus Consulting Private Limited, we are seeking an experienced and highly skilled Senior Data Engineer - Big Data Pipeline Specialist to join our team. This role will involve designing, developing, and implementing large-scale data pipelines using various technologies such as Spark,...
-
Big Data Specialist
1 month ago
Nagpur, India Taggd Full timePosition:Senior Cloud Data Engineer – AWS & Big DataLocation:Nagpur/PuneExperience: 7 to 9 yearsNotice Period :30 DaysTravel:Not required however the candidate is supposed to work with global teams in a remote environmentCompensation : 26 Lacs Annually ((This is the fixed compensation for this role. Candidates with salary expectations exceeding this amount...
-
Senior Engineer
2 weeks ago
Pune, India Infinity career edge Full timeResponsibilities :- Works closely with the business owner / Product owner / Cross functional Team and gathering technical requirements.- Experience in building and maintaining reliable and scalable ETL pipeline on big data & / or Cloud platform through the collection, storage, processing, and transformation of large datasets.- Collaborate with other teams to...
-
Senior Big Data Engineer
2 months ago
Pune, India AEXONIC TECHNOLOGIES PRIVATE LIMITED Full timePosition - Senior Big Data Engineer (GCP)Experience - 5 to 8 YrsLocation - RemoteShift - EST Shift 5:30 PM to 2:30 AMWe are seeking a highly skilled Senior Big Data Engineer with extensive experience in Big Data technologies, including Hadoop, Hive, PySpark, and expertise in Google Cloud Platform (GCP) and BigQuery. The successful candidate will be...
-
Senior Data Engineer for Big Data Platforms
3 weeks ago
Pune, Maharashtra, India Tata Consultancy Services Full timeWe are seeking an experienced Senior Data Engineer to work on our Big Data projects. The ideal candidate will have good work experience on platforms like Hadoop, Spark, Scala, Hive, Impala, and SQL.A successful candidate will have hands-on experience working on Data Engineering projects and a good understanding of SQL concepts.The role requires strong...
-
Senior Big Data Engineer
1 week ago
Pune, India AEXONIC TECHNOLOGIES PRIVATE LIMITED Full timeAre you ready to take your Big Data expertise to the next level? Join our dynamic team at Aexonic Technologies Pvt. Ltd. as a Senior Big Data Engineer (GCP) and work on cutting-edge projects in a collaborative and innovative environment.Role : Senior Big Data Engineer (GCP) Experience : 5+ years in Big Data technologies Shift : EST Shift (5:30 PM - 2:30 AM...
-
Senior Big Data Engineer
1 week ago
Delhi, Delhi, India CodeVyasa Full timeJob DescriptionWe are seeking a highly skilled Senior Big Data Engineer to join our team in Bangalore. As a key member of our data engineering team, you will be responsible for designing, implementing, and maintaining large-scale data processing pipelines using PySpark for both streaming and batch processing.Key Responsibilities:- Design, implement, and...
-
Pune, Maharashtra, India PubMatic Full timeAbout the RoleWe are seeking an experienced Senior Software Development Engineer to join our team in designing, developing, and testing complex engineering projects. The ideal candidate will have a strong background in software development, testing methodologies, and big data technologies.Key ResponsibilitiesDevelop automation frameworks using Python to...
-
Big Data Expert for Cloud Solutions
2 weeks ago
Ahmedabad, Gujarat, India AEXONIC TECHNOLOGIES PRIVATE LIMITED Full timeSenior Big Data Engineer Job DescriptionAbout AEXONIC TECHNOLOGIES PRIVATE LIMITEDAEXONIC TECHNOLOGIES PRIVATE LIMITED is seeking a highly skilled Senior Big Data Engineer to join our team. As a Senior Big Data Engineer, you will play a key role in designing, building, and maintaining robust data pipelines that support large-scale data processing and...
-
Senior Big Data Engineer
12 hours ago
Delhi, Delhi, India Knack Consulting Services Pvt Ltd. Full timeAbout the RoleWe are seeking a highly skilled Senior Big Data Engineer with expertise in Databricks to join our team at Knack Consulting Services Pvt. Ltd.
-
Senior Data Engineer
1 week ago
Pune, India RS Consultants Full timeJob Title : Senior Data Engineer. Location : Pune. Experience Level : 3+ Years. Company Type : AI/ML startup. Please apply only if you are comfortable working with Early stage startup. Company Overview : Join our client a pioneering startup founded by renowned tech experts, marking their second venture in the AI and ML field. The founders previously...
-
Senior Data Engineer
2 months ago
Pune, India RS Consultants Full timeJob Title : Senior Data Engineer. Location : Pune. Experience Level : 3+ Years. Company Type : AI/ML startup. Please apply only if you are comfortable working with Early stage startup. Company Overview : Join our client a pioneering startup founded by renowned tech experts, marking their second venture in the AI and ML field. The founders previously...