Big Data Engineer

16 hours ago

Bhavnagar, Gujarat, India beBeeDataiku Full time ₹ 18,00,000 - ₹ 25,00,000

Job Title: Data Science and Engineering Professional

Overview of the Role:

We are seeking a skilled professional to lead our Big Data Platform efforts. The ideal candidate will have extensive experience in managing big data platforms, handling day-to-day operations, and troubleshooting complex issues.

Key Responsibilities:
  • Administer and monitor Big Data Platform components including Hadoop, Hive, HBase, and BigSQL
  • Support technical and application team requests for data copies across environments, data cleanup, query tuning, etc.
  • Troubleshoot and resolve issues related to user queries, application jobs, etc.
  • Monitor and troubleshoot Big Data Platform, including hosted DBs, tools, and services
  • Creating encryption zones, managing encryption keys, and copying data between the encryption zones
  • Support projects and initiatives related to data acquisition, processing, and utilization through Big Data Platform
  • Maintain and support Big Data Platform CDP Private Cloud and IBM BigSQL across multiple environments
  • Install new versions, patches, support upgrades of CDP Private Cloud and IBM BigSQL
  • Monitor and troubleshoot platform issues and performance tuning
  • Support security policies, documentation, and compliance
  • Document processes and policies for Hadoop System Administration Platform support activities
  • Work with end-users to identify and resolve issues
  • Work with vendors to resolve issues as needed
  • Provide governance and support to project teams
  • Upgrade to CDP Private Cloud to CDP Public Cloud
Requirements:
  • Bachelor's or Master's degree in Computer Science or applicable field of study
  • 35 years of experience in Hadoop Ecosystem, HDFS, MapReduce, Hive, HBase, Spark, Scala, Yarn, Sqoop, Ranger, Knox, Kerberos, LDAP, SOLR
  • 6 years of experience with Java and Linux/Unix
  • 4 years of experience with Security Scripting and Automation
  • 2 years of experience in R, Python, Flume, Storm, Kafka
  • 2 years of experience with IBM BigSQL or similar MPP SQL Engines
  • 2 years of experience with MySQL
  • 1 year of experience with CICD
  • Experience with administration and performance tuning of Hadoop distributions, open source and commercial
  • Highly competent with Data Lake concepts and Big Data capabilities
  • Good understanding of data virtualization, replication, and Data Governance
  • Experience with AWS or other Cloud-based PaaS/SaaS environments
  • Experience with AWS EMR
  • Strong communication skill
  • Mandatory Skills: Dataiku, Hadoop, Hive, HBase, BigSQL, AWS, EMR

  • Advanced Big Data Engineer

    5 days ago

    Bhavnagar, Gujarat, India beBeeData Full time ₹ 15,00,000 - ₹ 20,12,000

    Job DescriptionWe are seeking a seasoned Big Data Developer to drive the design, implementation, and maintenance of large-scale data processing systems.The ideal candidate will have 2+ years of experience in Big Data development with expertise in Scala, AWS, and Python or Java.Design and implement scalable data processing systems using Big Data...

  • Data Engineer for Big Data Solutions

    1 week ago

    Bhavnagar, Gujarat, India beBeeInformatica Full time ₹ 1,00,00,000 - ₹ 1,50,00,000

    Job DescriptionThe ideal candidate will have a proven track record of designing and developing scalable ETL workflows using Informatica BDM/DEI.They will integrate data from structured and unstructured sources, including big data platforms, and apply strong data warehousing concepts to ensure efficient data modeling and transformation.Collaboration with data...

  • Chief Data Architect – Big Data Platforms

    2 weeks ago

    Bhavnagar, Gujarat, India beBeeBigDataLeader Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Lead Big Data Engineering TeamWe are seeking an experienced Lead to spearhead a team of data engineers working on large-scale distributed data systems. The ideal candidate will have deep expertise in Big Data platforms and cloud technologies.Key Responsibilities:Grow and lead a team of data engineers to design, develop, and deliver scalable data pipelines...

  • Big Data Architect

    13 hours ago

    Bhavnagar, Gujarat, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data ArchitectLooking for a challenging opportunity to leverage your technical expertise and drive business growth? We are seeking a talented Senior Data Architect to design and implement cutting-edge data platforms that will help our customers measure, communicate and eliminate cyber risks.This is an excellent chance to be part of a team building a...

  • Big Data Specialist

    1 week ago

    Bhavnagar, Gujarat, India beBeeClustering Full time ₹ 12,00,000 - ₹ 15,00,000

    Job Title: Big Data SpecialistWe are seeking an experienced professional with a strong background in big data administration to lead our team.Design, build, and manage complex big data architectures to ensure high performance and uptime.Migrate large datasets between different platforms while minimizing downtime.Perform performance tuning and optimization of...

  • Data Scientist with Big Data Expertise

    1 week ago

    Bhavnagar, Gujarat, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: Data Scientist with Big Data ExpertiseWe are seeking a skilled and experienced data scientist to join our team.The ideal candidate will have expertise in big data, including data ingestion from disparate sources onto big data platforms.Key Skills:PySparkHiveBenefits:A chance to work on exciting big data projectsA collaborative and dynamic work...

  • Big Data Expert

    4 days ago

    Bhavnagar, Gujarat, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title:Big Data Analyst and DeveloperJob Description:We are seeking a skilled Big Data specialist to design, develop, and implement data pipelines using latest technologies such as PySpark, Scala Spark, and Java Spark SQL. The ideal candidate will have experience in data structures and algorithms, with strong understanding of data transformation logic.The...

  • Big Data Platform Architect

    1 week ago

    Bhavnagar, Gujarat, India beBeeDataLeader Full time ₹ 25,00,000 - ₹ 35,00,000

    Job DescriptionSigmoid empowers enterprises to make smarter, data-driven decisions by blending advanced data engineering with AI consulting. We collaborate with leading organizations across sectors such as retail, finance, life sciences, and manufacturing to solve complex business challenges. Our global team specializes in cloud data modernization,...

  • Senior Big Data Solutions Architect

    2 weeks ago

    Bhavnagar, Gujarat, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Big Data ArchitectWe are seeking a skilled Big Data Architect to lead our data-driven initiatives and design scalable data pipelines. This role involves collaborating with data scientists and business stakeholders to deliver high-quality solutions.Key Responsibilities:Design, build, and optimize data pipelines and ETL workflows using Apache Spark, Python,...

  • Big Data Specialist

    1 week ago

    Bhavnagar, Gujarat, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Engineering SpecialistLocation: Flexible nationwide remote or hybrid, choice of the candidate.Experience: 7–10 Years (Minimum 3+ years in Scala)Notice Period: Immediate to 30 DaysMode of Work: Hybrid We are looking for a highly skilled Big Data Engineer with strong expertise in Scala, Spark, Python, NiFi, and Apache Kafka. The ideal...