Senior Big Data Engineer
1 day ago
Position : Senior Big Data Engineer (GCP)
Experience : 5+ Yrs
Shift Time : EST Shift 5 : 30 PM to 2 : 30 AM
Remote / Hybrid - Full Time
We are seeking a highly skilled Senior Big Data Engineer with extensive experience in Big Data technologies, including Hadoop, Hive, PySpark, and expertise in Google Cloud Platform (GCP) and BigQuery. The successful candidate will be responsible for designing, building, and maintaining robust data pipelines that support large-scale data processing and analytics. This role requires excellent data analysis skills and the ability to architect solutions for cloud-based environments, with a focus on extracting, processing, and analyzing data from diverse sources.
Key Responsibilities :
- Design, develop, and optimize Big Data pipelines and workflows using Hadoop, Hive, PySpark, and Google BigQuery.
- Architect data solutions to facilitate smooth data migration and processing between on-premises systems and Google Cloud Platform (GCP).
- Create and manage Unix Shell Scripts for process automation and data handling tasks.
- Perform in-depth data analysis and work comfortably with large datasets to derive actionable insights.
- Collaborate effectively with cross-functional teams to understand data requirements and translate them into scalable data solutions.
- Ensure data quality, integrity, and security across all stages of the data lifecycle.
- Stay updated with the latest industry trends and emerging technologies to continuously improve data solutions.
Mandatory Qualifications :
- Minimum of 5 years of industry experience working with Big Data technologies.
- Proficiency in Hadoop, Hive, and PySpark for data processing.
- Strong experience with Unix Shell Scripting.
- Expertise in SQL and extracting data from multiple sources.
- Demonstrated experience in data analysis, specifically querying and analyzing large datasets on Hadoop HDFS using Hive and Spark.
- Proven experience in architecting Big Data pipelines and data solutions.
- Proficiency in Google BigQuery and migrating data pipelines to GCP.
- Exceptional communication skills for clear requirement gathering and collaboration.
Preferred Qualifications :
- Google Cloud Platform (GCP) certification.
- Professional experience with cloud hosting platforms, with a preference for GCP.
-
Big Data
6 months ago
Remote, India Technology Next Full time**Big Data AI/ML Engineer** **Experience**: 6 to 9 Yrs **Location**:Remote **Tenure: - **6 Months **Salary: - ** 80K-90K In hand **Notice**:Immediate Joiner We are seeking a talented Data Engineer with AI & ML knowledge to join our team. As Data Engineer or ML Ops Engineer, your primary responsibility will be to develop & integrate ML solutions that...
-
Protegrity Engineering Manager
2 months ago
Remote, India PROTEGRITY INDIA PRIVATE LIMITED Full timeJob Title: Protegrity Engineering Manager - Big DataJob Summary:We are seeking a highly skilled Engineering Manager to lead our Big Data team at Protegrity. As a key member of our engineering team, you will be responsible for managing the development of our Big Data products and ensuring they meet the highest standards of quality and performance.Key...
-
Architect - Big Data
6 months ago
Remote, India Databricks Full timeCSQ125R88 As an **Architect (Big Data) **in our Professional Services team you will work with clients on short to medium term customer engagements on their **Big Data challenges using the Databricks platform**. You will provide data engineering, data science, and cloud technology projects which require integrating with client systems, training, and other...
-
Big Data Architect
6 months ago
Remote, India Databricks Full timeCSQ225R24 As a** Resident Solutions Architect** in our Professional Services team you will work with clients on short to medium term customer engagements on their **big data challenges using the Databricks platform**. You will provide data engineering, data science, and cloud technology projects which require integrating with client systems, training, and...
-
Architect : Big Data
6 months ago
Remote, India Databricks Full timeCSQ225R23 As a** Resident Solutions Architect (Big Data)**, you will work with clients on short to medium term customer engagements on their big data challenges using the Databricks platform. You will provide data engineering, data science, and cloud technology projects which require integrating with client systems, training, and other technical tasks to...
-
Big Data Architect
6 months ago
Remote, India Databricks Full timeCSQ424R34 As a** Big Data Architect **in our Professional Services team you will work with clients on short to medium term customer engagements on their **Big Data challenges using the Databricks platform**. You will provide data engineering, data science, and cloud technology projects which require integrating with client systems, training, and other...
-
Big Data
6 months ago
Remote, India KEG HR Services Full time**Greetings From KEG HR Services!** Job Role: Big Data & Kubernetes Administrator Experience: 3 Years to 5 Years **Job Description**: - Install, configure, and maintain Cloudera Big Data clusters across multiple environments, ensuring optimal performance and resource utilization. - Ensure high availability and manage the performance of data services like...
-
Senior Azure Data Engineer
1 week ago
Remote, India Quantumbricks Full timeJob Overview : We are seeking a highly skilled and experienced Senior Azure Data Engineer to join our team. The ideal candidate will have over 10 years of experience in designing, developing, and implementing data solutions on Azure. This role involves working closely with cross-functional teams to build scalable, efficient, and secure data platforms. You...
-
Data Engineering Lead
1 day ago
Remote, India AEXONIC TECHNOLOGIES PRIVATE LIMITED Full timeJob OverviewAEXONIC TECHNOLOGIES PRIVATE LIMITED is seeking a seasoned Senior Big Data Engineer to spearhead the development of cloud-based data pipelines and solutions.Salary & BenefitsThe estimated annual salary for this position is approximately $250,000. We offer a comprehensive benefits package, including medical insurance, retirement plan, and generous...
-
Azure Data Architect
6 months ago
Remote, India Data PlatformExperts Full time**Job Description: Azure Data Architect** **Key Responsibilities**: - **Design and Implementation**: - Design and implement end-to-end data solutions (data models, pipelines, data lakes, data warehouses) in Azure. - Optimize and maintain existing Azure data structures and integration processes. - Ensure architectural solutions are scalable, maintainable,...
-
Senior AWS Data Engineer
1 week ago
Remote, India Quantumbricks Full timeJob Summary. We are seeking a highly skilled and experienced Senior AWS Data Engineer to lead our cloud data architecture, design, and development initiatives. The ideal candidate will have deep expertise in building and managing large-scale data pipelines and solutions on AWS, as well as strong experience in data architecture, ETL, and real-time data...
-
AI Innovator for Big Data Solutions
6 days ago
Remote, India Enfec Solutions Full timeEnfec Solutions is a leading IT company specializing in big data, mobile app development, quality assurance, and business strategy. With over ten years of industry experience on average, our team ensures high-quality results.About the RoleWe are seeking an Artificial Intelligence Developer to join our team as a key contributor in developing innovative...
-
Senior Data Engineer
1 month ago
Anywhere in India/Multiple Locations/Remote Hunarstreet Technologies Pvt Ltd Full timeJob Title : Senior Data Engineer (Remote) Location : RemoteWe are hiring a Senior Data Engineer for one of our leading software development clients. The Senior Data Engineer will play a key role in building and extending data pipeline architecture, optimizing data flow, and supporting cross-functional teams. The ideal candidate has extensive experience in...
-
Big Data Architect
2 months ago
Remote, India Gcube Info Full time**Key Responsibilities** - **Data Architecture Design**: Lead the creation of scalable, secure, and high-performance data architectures to support business intelligence, reporting, and analytics. - **Data Strategy Development**: Develop, implement, and maintain the bank’s data strategy, aligning it with business objectives and regulatory requirements. -...
-
Content Writer and Editor
2 months ago
Remote, India Big Shout Design Full time**About Big Shout Design** Big Shout Design is a leading UI/UX design and digital marketing agency dedicated to delivering high-impact solutions. We believe in authentic storytelling and producing content that resonates with our audience. We are looking for a skilled **Content Writer** who is passionate about crafting compelling, AI-free, plagiarism-free,...
-
Big Data Engineer
1 month ago
Anywhere in India/Multiple Locations/Remote Hum Technologies Full timeBig Data Engineer with strong exp in Hadoop, Spark, SQL & AWS for one of the Fortune 500 Clients Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : : Apache NiFi, HDFS, Hive, Impala, Spark, Spark Streaming, Kafka, Java/Scala, SQL, Shell scripting Roles & Responsibilities : - Responsible for designing and implementing...
-
Azure Data Architect
6 months ago
Remote, India Data PlatformExperts Full timeJob Description: Azure Data Engineer (Architect Level) We are seeking an experienced and highly skilled Azure Data Engineer at the Architect level to lead the design and implementation of complex data solutions on the Microsoft Azure platform. As an Azure Data Engineer Architect, you will be responsible for providing technical expertise, guiding the...
-
Senior Data Engineer
6 months ago
Remote, India Nthinsight Full timeJob Description : We are seeking a skilled and experienced Data Engineer with expertise in Python, data warehousing, ETL (Extract, Transform, Load), and SQL. As a Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure to support the organization's data needs. You will collaborate with...
-
Data Engineering
1 week ago
Remote, India VSHR UNNATI LLP Full timeSALARY : 10LPA - 16LPA Job Overview :Our client is seeking an experienced and detail-oriented Data Engineering and Machine Learning Tester to join our Quality Assurance team. The ideal candidate will have strong expertise in testing data pipelines, machine learning models, and large-scale data processing systems, specifically within the Databricks...
-
Senior Data Engineer
1 week ago
Remote, India Quantumbricks Full timeJob Summary : We are seeking a highly skilled Senior Data Engineer with over 10 years of experience in designing, developing, and managing robust data pipelines, with specific expertise in Informatica Intelligent Cloud Services (IICS). The ideal candidate will have extensive experience in handling complex ETL/ELT processes, cloud data integration, and data...