Big Data admin-Hadoop, Spark, Nifi, Impala

2 days ago

Pune, Maharashtra, India Cortex Consultants Full time ₹ 15,00,000 - ₹ 25,00,000 per year
Work Experience: 4 to 6 years Official Notice Period – Immediate to 15 days Excellent communication skills Job Location – PuneShift Timing: Regular shift & UK shift (Should be flexible)
Job Description The RolePlan, manage, and oversee all aspects of a Production Environment for Big Data Platforms.Define strategies for Application Performance Monitoring, Optimization in Prod environmentRespond to Incidents and improvise platform based on feedback and measure the reduction of incidents over time.Ensures that batch production scheduling and process are accurate and timely.Able to create and execute queries to big data platform and relational data tables to identify process issues or to perform mass updates, preferred.Performs ad hoc requests from users such as data research, file manipulation/transfer, research of process issues, etc.Take a holistic approach to problem solving, by connecting the dots during a production event through the various technology stack that makes up the platform, to optimize meantime to recover.Engage in and improve the whole lifecycle of services—from inception and design, through deployment, operation and refinement.Analyze ITSM activities of the platform and provide feedback loop to development teams on operational gaps or resiliency concerns.Support services before they go live through activities such as system design consulting, capacity planning and launch reviews.Support the application CI/CD pipeline for promoting software into higher environments through validation and operational gating, and lead in DevOps automation and best practices.Maintain services once they are live by measuring and monitoring availability, latency and overall system health.Scale systems sustainably through mechanisms like automation and evolving systems by pushing for changes that improve reliability and velocity.Work with a global team spread across tech hubs in multiple geographies and time zones.Ability to share knowledge and explain processes and procedures to others. RequirementsExperience in Linux and Knowledge on ITSM/ITIL.Experience in the Big Data technologies (Hadoop, Spark, Nifi, Impala)2+ years of Experience in running Big Data production systems.Good to have experience in industry standard CI/CD tools like Git/BitBucket, Jenkins, Maven,Solid grasp of SQL or Oracle fundamentalsExperience with scripting, pipeline management, and software design.Systematic problem-solving approach, coupled with strong communication skills and a sense of ownership and drive.Ability to help debug and optimize code and automate routine tasks.Ability to support many different stakeholders. Experience in dealing with difficult situations and making decisions with a sense of urgency is needed.Appetite for change and pushing the boundaries of what can be done with automation.Experience in working across development, operations, and product teams to prioritize needs and to build relationships are a must.Experience designing and implementing an effective and efficient CI/CD flow that gets code from dev to prod with high quality and minimal manual effort is desired.Good Handle on Change Management and Release Management aspects of Software

  • Senior Spark Developer

    2 weeks ago

    Pune, Maharashtra, India Consulting Pandits Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Spark Rockstar Wanted Imagine a world where big data is not just a buzzword, but a playground where you get to build innovative solutions that make a real impact. We're on the hunt for a Senior Big Data Engineer who's passionate about Spark and has the skills to prove it. If you're a Spark aficionado with a knack for designing and developing scalable data...

  • Sr. Spark Big Data Developer

    1 day ago

    Pune, Maharashtra, India CRUTZ LEELA ENTERPRISES Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Sr. Spark Big Data DeveloperPFB the detailed JD:�� Experience: 5 to 7+ years (Hands-on)�� Notice Period: 0-30 DaysMust Have: Proficiency in at least one of the following programming languages:Java/Scala/Python Good understanding of SQL Experience of development and deployment of at least one end-to-end datastorage/processing pipeline...

  • Data Engineer

    1 day ago

    Pune, Maharashtra, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Title :- Data Engineer - PysparkExperience: 5 to 8 YearsLocation: Pune/HyderabadJob DescriptionRequired Skills:5+ years of experience in Big data and pysparkMust-HaveGood work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQLGood-to-HaveGood Spark, Pyspark,Big Data experienceSpark UI/Optimization/debugging techniquesGood...

  • Big Data Engineer

    3 days ago

    Pune, Maharashtra, India Coforge Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Coforge Ltd is Hiring for Big Data Engineer – AWS, Spark & Scala.Must Have Skills:- AWS, Spark & Scala.Experience Required: 3 to 6 YearsJob Locations: Pune, Hyderabad, Greater Noida Only.Send your CV to: For queries, contact via WhatsApp: Key Responsibilities:-• Design, develop, and optimize Big Data architectures leveraging AWS services for large-scale,...

  • Big Data Architect

    2 days ago

    Pune, Maharashtra, India Datametica Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title: Big Data Architect GCPExperience: 10–12 YearsLocation: PuneRoleWe are looking for a Big Data Architect with deep GCP expertise to define, design, and lead cloud-native data platforms for enterprise clients. You will be responsible for architecting large-scale data lakes, warehouses, and pipelines on GCP, modernizing on-premise workloads, and...

  • Bigdata - Spark

    1 day ago

    Pune, Maharashtra, India Virtusa Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    7+ years of experience in Big Data with strong expertise in Spark and ScalaMandatory Skills\: Big Data Primarily Spark and ScalaStrong Knowledge in HDFS, Hive, Impala with knowledge on Unix , Oracle, Autosys, Good to Have \: Agile Methodology and Banking ExpertiseStrong Communication SkillsNot limited to Spark batch, need Spark streaming experience No SQL...

  • Big Data Developer

    1 day ago

    Pune, Maharashtra, India Uplers Full time ₹ 60,000 - ₹ 1,80,000 per year

    Experience: yearsSalary: Confidential (based on experience)Expected Notice Period: 15 DaysShift: (GMT+05:30) Asia/Kolkata (IST)Opportunity Type: Office (Pune)Placement Type: Full Time Permanent position(Payroll and Compliance to be managed by: Nexus Cognitive)**(*Note: This is a requirement for one of Uplers' client - Nexus Cognitive)**What do you need for...

  • Hadoop Admin

    1 day ago

    Pune, Maharashtra, India CRUTZ LEELA ENTERPRISES Full time ₹ 2,00,000 - ₹ 10,00,000 per year

    Position - Hadoop AdminExperience: 4 to 7 YearsJob Description: -4+ years of hands-on experience - Hadoop, System administration with sound knowledgeIn Unix based Operating System internals. Working experience on Cloudera CDP and CDH and Hortonworks HDP Distribution. Linux experience (RedHat, CentOS, Ubuntu). Experience in setting up and supporting...

  • Big Data Engineer

    1 week ago

    Pune, Maharashtra, India FUNIC TECH PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Description :Job Opening : Big Data Engineer (7-10 yrs)Location : Aundh, Pune (Work From Office)Job Type : Full-Time | General Shift | Monday FridayKey Responsibilities : - Build and manage highly scalable Big Data pipelines for large structured datasets. - Collaborate with Product Management & Engineering leadership to design and implement optimal...

  • Big Data Engineer

    1 week ago

    Pune, Maharashtra, India Funic Tech Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Opening: Big Data EngineerLocation: Aundh, Pune (Work From Office)Job Type: Full-Time | General Shift | Monday – FridayKey ResponsibilitiesBuild and manage highly scalable Big Data pipelines for large structured datasets.Collaborate with Product Management & Engineering leadership to design and implement optimal solutions.Participate in design...