Big Data Expert

3 days ago

India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000
Job Opportunity

DeltaCubes is seeking a Senior Data Engineer to fill a key position. This role involves designing and developing large-scale data pipelines, as well as utilizing Google Cloud tools and technologies.


Key Responsibilities:
  • Design, develop, test, and maintain data acquisition pipelines for structured and unstructured data
  • Create complex datasets based on business requirements
  • Construct big data pipeline architecture
  • Identify opportunities for data acquisition through collaboration with stakeholders and business clients
  • Translate business needs into technical requirements
  • Utilize various tools in the Google Cloud Ecosystem, including Python, Data Flow, Data Stream, CDC, Cloud Functions, Cloud Run, Pub Sub, Big Query, and Cloud Storage, to integrate systems and data pipelines
  • Monitor pipelines using logs and alerts
  • Replicate SAP tables to Google Cloud using SLT
  • Develop JSON messaging structure for integrating with various applications
  • Leverage DevOps and CI/CD practices (GitHub, Terraform) to ensure the reliability and scalability of data pipelines
  • Partition, cluster, and retrieve content in Big Query while securing data with IAM roles and Policy Tags
  • Secure access to datasets using roles and authorized views
  • Design and build an ingestion pipeline using REST API

Requirements include at least 4+ years of experience in a GCP data engineering role, with a strong understanding of Google Cloud tools and technologies. Experience with DevOps and CI/CD practices, as well as proficiency in SQL and Python, is highly desirable.

  • Big Data Architect

    6 days ago

    India Bloom Consulting Services Full time

    Not Disclosed Experience **12 - 16 Years** Offered Salary **Lakh** Notice Period **Not Disclosed** **Role: Big Data Architect** **Location: Bangalore** **Skills required**: Total 12 to 16 years of experience in BI, analytics, visualization, **data** integration, designing and developing **data **warehouses (ETL, **data**bases, front-end...

  • Big Data Architect

    6 days ago

    India Bloom Consulting Services Full time

    NA Experience **12 - 16 years** Offered Salary Notice Period **Not Disclosed** **Role: Big Data Architect** **Location: Bangalore** **Skills required**: Total 12 to 16 years of experience in BI, analytics, visualization, **data** integration, designing and developing **data **warehouses (ETL, **data**bases, front-end tools). 9+ years of...

  • Senior Big Data Engineering Expert

    3 days ago

    India beBeeBigData Full time ₹ 8,00,000 - ₹ 12,00,000

    Job Title: Big Data Engineer Lead">Job DescriptionWe are seeking a skilled and experienced Big Data Engineer to lead our team in the implementation of big data solutions. As a Big Data Engineer Lead, you will be responsible for designing, developing, and deploying large-scale data processing systems.You will work closely with cross-functional teams to...

  • Senior Big Data Analyst

    7 days ago

    India beBeeData Full time ₹ 9,00,000 - ₹ 12,00,000

    Job OpportunityAbout the Role:Are you a skilled professional seeking a challenging role that leverages your expertise in big data and analytics? Look no further!Responsibilities:Develop and implement big data solutions using Spark, SQL, Hive, and Scala.Collaborate with cross-functional teams to design and deliver high-quality data-driven insights.Stay...

  • Big Data Systems Expert

    2 weeks ago

    India beBeeHadoop Full time ₹ 15,00,000 - ₹ 20,10,000

    Job Title: Hadoop Cluster AdministratorJob Summary:We are seeking a highly skilled Hadoop Cluster Administrator to manage and maintain our big data infrastructure. The ideal candidate will have experience in designing, building, and optimizing Hadoop clusters for high-performance data processing.Responsibilities:Design and implement scalable Hadoop cluster...

  • Big Data Engineer

    6 days ago

    India Hexaware Technologies Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    Job description for Data engineer: Education/Experience: • Bachelor's degree in computer science or equivalent training required. Big Data Engineer: Years Experience in developing Big Data processing applications on Azure Databricks. 2. Profound experience with developing Big Data applications using Spark and Scala/Python. 3. Good experience in working...

  • Big Data Administrator

    3 days ago

    India HNM Solutions Full time

    **Big Data Administrator**: - (Engagement - 6 Months+) _ **Experience**:5+ years **Key Skills**:Hadoop ecosystem (HDFS, YARN, Hive, Impala), Cloudera/HDP administration, Linux, security, monitoring **Responsibilities**:Administer and monitor big data infrastructure & jobs, manage user access, data storage, and cluster health; implement security controls...

  • Big Data Architect

    7 days ago

    India Persistent Systems Full time

    **About Persistent** We are a trusted Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what’s next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with...

  • Big Data Developer

    1 day ago

    india Tata Consultancy Services Full time

    4+ years of hands on development experience in programming languages such as JAVA,SCALA using Maven, Apache Spark Frameworks and Unix Shell scripting Should be comfortable with Unix File system as well as HDFS commands Should have worked on query languages such as Oracle SQL, Hive SQL, Spark SQL, Impala, HBase DB Should be flexible Should have good...

  • Big Data Engineer

    1 week ago

    India beBeeDataEngineering Full time ₹ 12,00,000 - ₹ 19,00,000

    Job Title: Big Data EngineerKey Responsibilities:Design, build, and maintain large-scale data processing systems.Develop PySpark applications using Spark Dataframes in Python.Optimize spark jobs to process huge volumes of data.Utilize Amazon's Analytics services such as EMR, Athena, and AWS Glue.Work with version control tools like Git and AWS...