Big Data Engineer
1 month ago
Position - Big Data Engineer
Location - Pune/Hyderabad/Any
Experience - 5+
Job Summary
- As a Big data engineer responsibilities include - review and analysis of requirements, design workshop participation and facilitation, design creation and documentation, data pipeline implementation, testing, bug fixing, release support and post release aftercare.
- You will be part of a team responsible for delivering the highest quality data processing and analytical systems, drawing upon your engineering and coding expertise, whilst being open minded to the opportunities the cloud provides. You will deliver code, facilitate solution designs and identify potential technical impediments then work to resolve them.
Roles & Responsibilities
- Devise technical solution options aligned to the strategic architecture.
- Design data models / structures to support capital modelling and reporting processes.
- Influence the design and implementation of batched processes to create model inputs and execute models
- Liaise with other projects to achieve synergies through mutually beneficial design / delivery decisions.
- Act as a technical mentor/coach to developers to help them deliver solutions that meet customer requirements and architectural standards.
- Be available to discuss and resolve technical issues with individuals within and without the team.
- Help to embed the use of new technologies to deliver project goals.
- Help to ensure work carried out is in line with quality standards
Must Have:
·Strong knowledge of Hadoop/Big Data concepts
·Experience in working with Cloudera clusters is a plus
·Experience in Analysing data sets and creating data models using SQL
·Good working knowledge of Apache Spark with Java/Scala
·Knowledge of Spark Streaming with Kafka
·Must have hands on experience in building ETL pipelines using Apache NiFi
·Must have hands on experience in Impala with Kudu
·Experience in writing transformations using SQL
·Experience in monitoring, debugging and troubleshooting Spark Jobs/Impala Queries in the cluster
·Basic knowledge of Unix shell scripting
·Good to have knowledge of Spring boot, Spring MVC, REST APIs and Microservices.
·Good to have knowledge of AWS Cloud (EC2, IAM, S3, Lambda, CloudWatch etc.)
Skills: Apache NiFi, HDFS, Hive, Impala, Spark, Spark Streaming, Kafka, Java/Scala, SQL, Shell scripting
-
GCP Data Engineers
4 weeks ago
Belgaum, India Rise& Shift Full timeJob Description for GCP Data Engineers with Rise and Shift We are looking for experienced data engineer with hands on experience on GCP offerings Experienced in BigQuery/ BigTable/ Pyspark. Worked on prior data engineering projects leveraging GCP product offerings. Strong SQL background. Data engineering pipeline build on big data on GCP. Role: GCP Data...
-
Data Engineer
4 weeks ago
Belgaum, India NucleusTeq Full timeCompany DescriptionNucleusTeq is a global software solutions and product company headquartered in Phoenix, Arizona. Specializing in Big Data, Analytics, Cloud Computing, Blockchain, Enterprise Automation, and Mobility, we empower businesses through digital innovation. Our diverse portfolio of Fortune 100 clients and global partners showcases our ability to...
-
Senior Data Engineer
9 hours ago
belgaum, India redBus Full timeredBus.in is India's largest online bus ticketing platform operating across India, SEA & LATAM (Part of GO-MMT group, Nasdaq: MMYT). We have the largest network of bus operators and satisfied customers booking upto 250,000 transactions a day through mobile and desktop channels. With over 300,000+ live routes in multiple continents we host the largest...
-
Data Engineering Manager
3 months ago
Belgaum, India Deutsche Telekom Digital Labs Full timeThe Role As an Engineering Manager, you will ensure consistency and quality by shaping the right strategies. You will keep an eye on all engineering projects and ensure all duties are fulfilled. You will also act as an Architect for this team and will be responsible for design and long-term roadmap. You will analyse other employees’ tasks and carry on...
-
Data Engineer
3 days ago
Belgaum, India L&T Technology Services Full timeAbout the CompanyWe are looking for a Data Bricks Engineer with Azure cloud experience.Data Engineer (Databricks + Azure Cloud )About the RoleWe are looking for individuals with 5+ years of relevant data engineering hands-on work experience in data ingestion, processing, and exploratory analysis to build solutions that deliver value through data as an asset....
-
Software Engineer/Data engineer
3 weeks ago
Belgaum, India Info Services Full timewe are looking for people who have developed and operated Airflow—not just used airflow to orchestrate data ResponsibilitiesBe a key player in a cross-functional team of engineers who build software for a large-scale data processing ecosystem, supporting real-time and batch data pipelines for analytics, data science, and operations.Build tools and services...
-
Machine Learning Engineer
1 month ago
Belgaum, India ValueLabs Full timeAbout the Company: ValueLabs is hiring for a contract job for an Azure Machine Learning Engineer. The company values advanced statistical analysis, predictive modeling, and machine learning.About the Role: The key responsibilities include proficiency in Python, SQL, and various libraries, as well as experience with the Azure ecosystem and Big Data...
-
Data Engineer
3 days ago
Belgaum, India HARMAN India Full timeHarman India is hiring Data Engineering experts!Job Location: BangaloreExperience: 4years- 8YearsNotice Period- Immediate to 15/20DaysMode: HybridRequired Skill:Bachelor's degree in computer science or related field (Master's preferred) with 4-8 years of overall work experience. 2. Expertise in ETL processes, data warehousing and data pipeline construction...
-
Data Architect Lead
4 days ago
Belgaum, Karnataka, India Varcus infinity Full timeJob Title: Senior Data EngineerAt Varcus infinity, we are seeking an experienced Senior Data Engineer to build and manage data pipelines that support our data programs, optimize processes, and drive advanced business intelligence.About the Role:We are looking for a highly skilled data engineer to design, implement, and maintain scalable data pipelines that...
-
Cyber Security Engineer
1 month ago
belgaum, India Quantiphi Full timeAbout The Company:Quantiphi is an award-winning AI-first digital engineering company driven by the desire to solve transformational problems at the heart of business. Quantiphi solves the toughest and complex business problems by combining deep industry experience, disciplined cloud and data-engineering practices, and cutting-edge artificial intelligence...
-
Data Analyst
3 days ago
Belgaum, India System Soft Technologies Full timeJob Description – Data AnalystSystem Soft Technologies is a premier technology company providing exceptional consulting services and solutions that drive innovation, enhance business value, and boost competitiveness. For over 25 years, we have built trusted partnerships with our clients, helping us grow into a $200MM+ enterprise. With the collective...
-
Data Science Professional
7 days ago
Belgaum, Karnataka, India Quantzig Full timeAbout QuantzigA global analytics and advisory firm with offices in the US, UK, Canada, China, and IndiaWe assist clients across the globe with end-to-end advanced analytics, visual storyboarding, Machine Learning, and data engineering solutions for prudent decision makingKey Responsibilities:Migrate use cases from Python or Databricks to a centralized...
-
Senior Director of Data Science
4 weeks ago
Belgaum, Karnataka, India CriticalRiver Inc. Full timeAbout CriticalRiver Inc.We are a trusted digital technology consulting company with a proven track record of successful technology implementations in areas such as Digital Transformation, Digital Experience Management, Digital Engagement, and Digital Engineering. With 100+ Digital Transformation engagements, we have established ourselves as a leader in the...
-
Azure AI Engineer
3 days ago
Belgaum, India CAVU Capital Full timeWe are looking for a full-time contract for a large client in the insurance industry.The role is a Microsoft Azure AI Engineer.Start Date is January, 2025.Azure AI EngineerAs a Microsoft Azure AI engineer, you build, manage, and deploy AI solutions that leverage Azure AI.Your responsibilities include participating in all phases of AI solutions development,...
-
DevOps Engineer with Azure Expertise
3 weeks ago
Belgaum, Karnataka, India SourceBae Full timeJob Title: DevOps Engineer with Azure ExpertiseLocation: RemoteExperience: 6+ yearsKey Skills: Classic Pipeline, PowerShell, YAML, Bicep, ARM Template/Terraform, CI/CD, Azure Data Lake & AnalyticsMost Important Skills: Data background with Azure & PowerShellJob Description:We are seeking a skilled DevOps Engineer with a strong background in Azure Data...
-
Design Engineer
4 weeks ago
Belgaum, India Expert Global Group Full timeDear Candidate,Greetings from Expert Global Solution Hope you’re doing well!!We have requirement for Sr. Designer, Hyderabad Experience: Min. 5 years of exp. Payroll: Expert Global Solution Pvt. Ltd.Position: Sr. DesignerJob Profile – MAIN TASKS:Prepares advanced CAD detail and assembly drawings from sketches, mark-up prints and other reference materials...
-
Backend Engineer in Generative AI
1 month ago
belgaum, India Stealth Full timeAbout UsWe are an innovative AI startup dedicated to revolutionizing industries with cutting-edge generative AI solutions. Our mission is to leverage advanced AI technologies to create impactful products that solve real-world problems. As we continue to grow, we are looking for a skilled Backend Engineer to join our team and help build the infrastructure...
-
Technical Consultant
3 weeks ago
Belgaum, India Rapid Circle Full timeMaking a difference and driving positive change is what we do every day at Rapid Circle. Our Cloud Pioneers help our clients in their digital transformation. Are you someone who goes for constant, positive change? Then this vacancy is for you! As a Cloud Pioneer at Rapid Circle, you will work with our customers on different projects. For example, making...
-
ITGC SOX Testing Specialist
3 days ago
Belgaum, Karnataka, India One Of The WorldS Big Four Audit & Consultancy Firm Full timeThe role involves working with one of the world’s biggest four audit and consultancy firms on a permanent basis. Relevant details are as follows:A. Job Location – BangaloreB. Area of Expertise – IT General Controls (ITGC) control testing in Oracle EnvironmentC. Experience – 5-8 YrsKey Responsibilities:• Test IT General Controls (Access/ Password...
-
Senior Piping Design Engineer
3 weeks ago
Belgaum, Karnataka, India Snaphunt Full timeThe OpportunityWe are seeking a highly skilled Senior Piping Design Engineer to join our team. As a key member of our engineering team, you will be responsible for conducting on-site visits to project locations, gathering data, and collaborating with stakeholders to understand site-specific requirements.Key Responsibilities:Develop and coordinate P&ID...