Big Data Production Systems Engineer
2 weeks ago
Fulcrum Digital is an agile and next-generation digital accelerating company providing digital transformation and technology services right from ideation to implementation.
Our team is looking for a skilled System Reliability Engineer to join our Big Data Platform team.
The ideal candidate will have experience in running Big Data production systems, with a strong understanding of technologies such as Hadoop, Spark, and Nifi.
We are offering a competitive salary of $140,000 per year, along with a comprehensive benefits package and opportunities for career growth and development.
Responsibilities:
- Plan, manage, and oversee all aspects of a Production Environment for Big Data Platforms
- Define strategies for Application Performance Monitoring, Optimization in Prod environment
- Respond to Incidents and improvise platform based on feedback and measure the reduction of incidents over time
- Ensures that batch production scheduling and process are accurate and timely
- Able to create and execute queries to big data platform and relational data tables to identify process issues or to perform mass updates, preferred
- Performs ad hoc requests from users such as data research, file manipulation/transfer, research of process issues, etc.
- Take a holistic approach to problem solving, by connecting the dots during a production event through the various technology stack that makes up the platform, to optimize meantime to recover
- Engage in and improve the whole lifecycle of services—from inception and design, through deployment, operation and refinement
- Analyze ITSM activities of the platform and provide feedback loop to development teams on operational gaps or resiliency concerns
- Support services before they go live through activities such as system design consulting, capacity planning and launch reviews
- Support the application CI/CD pipeline for promoting software into higher environments through validation and operational gating, and lead in DevOps automation and best practices
- Maintain services once they are live by measuring and monitoring availability, latency and overall system health
- Scale systems sustainably through mechanisms like automation and evolving systems by pushing for changes that improve reliability and velocity
Requirements:
- Experience in the Big Data technologies (Hadoop, Spark, Nifi,Impala)
- Experience in Linux & knowledge on ITSM/ITIL
- 5+ years of Experience in running BigData production systems
- Good to have experience in industry-standard CI/CD tools like Git/BitBucket, Jenkins, Maven, Artifactory, and Chef
- Solid grasp of SQL or Oracle fundamentals
- Experience with scripting, pipeline management, and software design. Systematic problem-solving approach, coupled with strong communication skills and a sense of ownership and drive
- Ability to help debug and optimize code and automate routine tasks. Ability to support many different stakeholders. Experience in dealing with difficult situations and making decisions with a sense of urgency is needed. Appetite for change and pushing the boundaries of what can be done with automation
- Experience in working across development, operations, and product teams to prioritize needs and to build relationships are a must
- Experience designing and implementing an effective and efficient CI/CD flow that gets code from dev to prod with high quality and minimal manual effort is desired
- Good Handle on Change Management and Release Management aspects of Software
-
Big Data Production Systems Manager
1 month ago
Pune, Maharashtra, India Fulcrum Digital Full timeJob Role OverviewFulcrum Digital is an agile and next-generation digital accelerating company providing digital transformation and technology services across various industries. As a key member of our team, you will play a pivotal role in ensuring the reliability and performance of our Big Data Platforms.Key ResponsibilitiesThe ideal candidate will be...
-
Data Quality Engineer for Big Data Systems
2 weeks ago
Pune, Maharashtra, India MICHELIN Full timeWe are seeking a seasoned Data Quality Engineer to join our MICHELIN team. As a key player in our data management and processing domain, you will be responsible for ensuring the quality of our data entering the Data Lake.In this role, you will oversee and integrate data from various sources, guaranteeing its accuracy and relevance for our Data Analysts and...
-
Big Data Engineer for Platform Optimization
7 days ago
Pune, Maharashtra, India Talentmatics Full timeJob DescriptionTalentmatics is seeking an experienced Big Data Engineer to optimize our big data platform in Pune and Hyderabad locations.We offer a competitive salary of ₹1,200,000 per annum, commensurate with experience, to manage and maintain our big data ecosystem.About the RoleThis is an exciting opportunity to work on secured environments using...
-
Big Data Automation Engineer
6 days ago
Pune, Maharashtra, India PubMatic Full timeAbout the Role:The ideal candidate will have 1-3 years of experience working as a Software Development Engineer in Test (SDET) and have done meaningful automation.A strong understanding of programming skills, with Python being preferred, is essential for this role.Hands-on experience in automating backend applications, such as databases and REST APIs, is...
-
Highly Scalable Big Data Engineer
4 days ago
Pune, Maharashtra, India Recro Full timeAbout the RoleWe are seeking a highly skilled Big Data Engineer to join our team at Recro.Job DescriptionIn this role, you will be responsible for designing and implementing scalable data pipelines using distributed processing tools like Akka and Spark.You will work closely with Product Management and Engineering leadership to devise and build the right...
-
Scalable Big Data Engineer
2 weeks ago
Pune, Maharashtra, India Virtusa Full timeJob Title: Scalable Big Data EngineerAbout the RoleVirtusa is seeking an experienced Scala and Spark Engineer to join our team. As a key member of our engineering team, you will be responsible for designing, developing, and maintaining large-scale data processing systems using Scala and Spark.Key Responsibilities- Design and develop scalable big data...
-
Big Data Solutions Architect
2 days ago
Pune, Maharashtra, India Mig Staffing Full timeAbout Mig StaffingWe are a leading staffing agency specializing in the placement of highly skilled professionals in various industries. Our team of experts works closely with clients to understand their unique needs and requirements.Job Title: Big Data Solutions ArchitectJob Summary:We are seeking an experienced Big Data Solutions Architect to join our team....
-
Senior Big Data Developer
3 weeks ago
Pune, Maharashtra, India NTT DATA Full timeCompany OverviewNTT DATA is a global technology services company that helps clients innovate, optimize and transform for long term success. We have diverse experts in more than 50 countries and a robust partner ecosystem of established and start-up companies.Our Commitment to YouEstimated Salary: ₹2,200,000 - ₹3,500,000 per annum (dependent on...
-
GCP Data Engineer
5 days ago
Pune, Maharashtra, India Tata Consultancy Services Full timeWe are looking for a highly skilled GCP Data Engineer - Big Data Expert to join our team at Tata Consultancy Services. As a GCP Data Engineer, you will play a critical role in designing and implementing scalable data processing systems on Google Cloud Platform.About the Role:We are seeking an experienced professional with a strong background in GCP-based big...
-
Pune, Maharashtra, India PubMatic Full timeAbout the RoleWe are seeking an experienced Senior Software Development Engineer to join our team in designing, developing, and testing complex engineering projects. The ideal candidate will have a strong background in software development, testing methodologies, and big data technologies.Key ResponsibilitiesDevelop automation frameworks using Python to...
-
Data Engineer
3 days ago
Pune, Maharashtra, India LTIMindtree Full timeJob DescriptionWe are seeking a highly skilled Data Engineer to join our team at LTIMindtree. As a Data Engineer, you will play a key role in designing and developing big data platforms using Scala and Spark.Responsibilities:Design and develop big data platforms using Scala and Spark.Develop expertise in Data Engineering, including hands-on experience in...
-
Data Analytics Engineer
2 weeks ago
Pune, Maharashtra, India PubMatic Full timePubMatic is an innovative technology company revolutionizing the digital advertising industry with its cutting-edge sell-side platform.The platform empowers leading digital content creators to control their inventory and maximize monetization by enabling marketers to drive return on investment and reach addressable audiences across ad formats and...
-
Data Engineer Leader
1 month ago
Pune, Maharashtra, India DATAECONOMY Full timeAbout the RoleDataEconomy is seeking a Senior Data Engineer to lead our team in designing and implementing big data solutions using PySpark.Key ResponsibilitiesLead the design and implementation of scalable and robust data pipelines using PySpark.Collaborate with cross-functional teams to understand business needs and develop data-driven solutions.Mentor and...
-
Senior Data Pipeline Engineer
3 days ago
Pune, Maharashtra, India Tata Consultancy Services Full timeAbout the RoleWe are seeking a highly skilled Senior Data Pipeline Engineer to join our team in Pune, Hyderabad, Bangalore, Chennai, Noida or Nagpur. The ideal candidate will have 5-10 years of experience in designing and building production data pipelines using Cloud Native GCP, Python, SQL and other relevant technologies.Key ResponsibilitiesDesign and...
-
Big Data Solutions Developer
2 weeks ago
Pune, Maharashtra, India Mig Staffing Full timeJob Title: Big Data Solutions DeveloperJob Description:Mig Staffing is seeking a Big Data Solutions Developer to design, implement, and manage our company's big data solutions. Key responsibilities include developing Hadoop systems, loading disparate data sets, conducting pre-processing services, and collaborating with software research and development...
-
Big Data Platform Specialist
3 days ago
Pune, Maharashtra, India Talentmatics Full timeTalentmatics is seeking an experienced Big Data Platform Specialist to join our team in Pune or Hyderabad.Estimated Salary: ₹2,500,000 - ₹4,200,000 per annum.Job Description:We are looking for a skilled Hadoop administrator with 5-8 years of work experience to manage our Big Data platform and allied toolset. The ideal candidate should have strong...
-
Big Data Engineering Specialist
4 days ago
Pune, Maharashtra, India LTIMindtree Full timeWe are seeking a highly skilled Big Data Engineering Specialist to join our team at LTIMindtree. As a key member of our big data engineering team, you will be responsible for designing and developing scalable big data platforms using modern technologies like Scala Spark and Hadoop ecosystem.This role requires a deep understanding of modern data processing...
-
Pune, Maharashtra, India Radical Technologies Full time**Job Title:** Cloud Computing Engineer with Big Data Experience**Company Overview:** Radical Technologies is a cutting-edge company that empowers innovation through data-driven solutions.**Salary:** ₹25,00,000 - ₹40,00,000 per annum based on experience and qualifications.**Job Description:We are seeking an experienced Cloud Computing Engineer to join...
-
Big Data Engineer
3 days ago
Pune, Maharashtra, India LTIMindtree Full timeJob SummaryWe are seeking a highly skilled Big Data Engineer - Databricks Expert to join our team at LTIMindtree. The ideal candidate will have experience with Azure Databricks, Python, and Time Series Processing.About the RoleAs a Big Data Engineer - Databricks Expert, you will design, construct, install, test, and maintain highly scalable data management...
-
Big Data Engineer
2 weeks ago
Pune, Maharashtra, India PubMatic Full timeCompany OverviewPubMatic is a leading digital advertising company that empowers the world's top content creators to control their inventory and increase monetization. With its sell-side platform, PubMatic delivers scalable and flexible programmatic innovation, improving outcomes for customers while championing a vibrant and transparent digital advertising...