Senior Big Data Engineer

18 hours ago

Hubli, Karnataka, India beBeeData Full time US$ 1,00,000 - US$ 2,00,000
Big Data Expert

Our client is seeking an experienced Big Data expert to join their team. The ideal candidate will have extensive knowledge of Big Data technologies and Google Cloud Platform services.

  • Design, develop and optimize data pipelines using GCP BigQuery Dataflow and Apache Airflow to support large-scale data analytics.
  • Utilize the Big Data Hadoop ecosystem to manage and process vast datasets efficiently.
  • Collaborate with cross-functional teams to gather requirements and deliver reliable data solutions.
  • Ensure data quality, consistency and integrity across multiple data sources.
  • Monitor and troubleshoot data workflows to maintain high system availability and performance.
  • Stay updated with emerging trends and best practices in GCP data analytics and big data technologies.
  • Implement and manage ETL processes leveraging GCP services such as BigQuery Dataflow and Airflow.
  • Develop scalable, maintainable and reusable data pipelines to support business intelligence and analytics needs.
  • Optimize SQL queries and data models for performance and cost efficiency in BigQuery.
  • Integrate Hadoop ecosystem components with GCP services to enhance data processing capabilities.
  • Automate workflow orchestration using Apache Airflow for seamless data operations.
  • Collaborate with data engineers, analysts and stakeholders to ensure alignment of data solutions with organizational goals.
  • Participate in code reviews, testing and deployment activities adhering to best practices.
  • Mentor junior team members and contribute to continuous improvement initiatives within the data engineering team.
  • The ideal candidate must have the following skills:
  • GCP Storage
  • GCP BigQuery
  • GCP DataProc
  • GCP Cloud Composer
  • GCP DMS
  • Apache airflow
  • Java
  • Python
  • Scala
  • GCP Datastream
  • Google Analytics Hub
  • GCP Workflows
  • GCP Dataform
  • GCP Datafusion
  • GCP Pub/Sub
  • ANSI-SQL
  • GCP Dataflow
  • Big Data Hadoop Ecosystem

Benefits include a competitive salary, excellent benefits package, opportunities for growth and professional development, and a dynamic work environment.

Why Join Our Team?

As a member of our team, you will have the opportunity to work on challenging projects, collaborate with talented professionals, and contribute to the success of our organization.

  • Big Data Engineering Expertise

    1 week ago

    Hubli, Karnataka, India beBeeDataEngineering Full time US$ 1,00,000 - US$ 1,50,000

    Job Title: Big Data Engineering ExpertiseAbout the Role:As a seasoned big data engineer, you will be responsible for designing, building, and maintaining high-performance data pipelines that process vast amounts of data.Key Responsibilities:

  • Senior Big Data Architect

    1 day ago

    Hubli, Karnataka, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Big Data Engineer PositionJob Summary:We are seeking a highly skilled Big Data Engineer to design and implement scalable data transformation pipelines using Snowflake and dbt, following industry-accepted best practices.Key Responsibilities:Data Pipeline Development: Design and implement robust ELT pipelines using dbt on Snowflake, ensuring scalability and...

  • Lead Big Data Engineer

    2 days ago

    Hubli, Karnataka, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Data Engineering RoleJob DescriptionGCP Data Engineers are responsible for designing, implementing, and managing data pipelines on Google Cloud Platform (GCP) to extract valuable insights from large datasets.Key ResponsibilitiesData Pipeline Design and Development: Design and develop scalable data pipelines using GCP services such as Cloud Dataflow, Cloud...

  • Big Data Systems Engineer

    11 hours ago

    Hubli, Karnataka, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 30,00,000

    Job Title:Data Engineering SpecialistAbout the Role:We are seeking a highly skilled Data Engineering Specialist to join our team. The ideal candidate will have a strong background in data engineering and experience with designing, building, and maintaining large-scale data systems.Key Responsibilities:Create and maintain events/streaming based...

  • Big Data Specialist

    18 hours ago

    Hubli, Karnataka, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Big Data Engineer - Job OpportunityWe are seeking a highly skilled professional with expertise in Big Data technologies and AWS Cloud Services. The ideal candidate will bring strong hands-on experience in designing, architecting, and implementing scalable data engineering solutions while driving innovation within the team.Design, develop, and optimize Big...

  • Senior Big Data Architect

    3 days ago

    Hubli, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,20,00,000

    Big Data Systems DeveloperWe are seeking a skilled Data Engineer II to join our team. This role is crucial in designing and executing large-scale distributed data systems.The ideal candidate should have 2 to 4 years of experience in big data technologies, such as Apache Hadoop, and relational databases, including MS SQL Server, Oracle, MySQL, and...

  • Big Data Solutions Architect

    2 weeks ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    About the RoleA highly skilled Big Data Engineer is required to lead a team of big data engineers. This individual will work closely with clients and team members to understand their requirements and develop architectures that meet their needs.Key Responsibilities:Support the Data Engineering team in setting up the Data Lake on Cloud and implementing...

  • Big Data Specialist

    3 days ago

    Hubli, Karnataka, India beBeeDataScientist Full time ₹ 60,00,000 - ₹ 1,20,00,000

    Job OverviewThe role involves working with large-scale data sets utilizing Big Data tools and technologies. We are looking for a skilled professional to join our team as a Data Scientist.Key Responsibilities:Develop, tweak queries, and work on performance enhancement using Scala, Spark, Hive, and Hadoop Big Data Eco System Tools.Apply SQL experience to...

  • Senior Engineering Manager – Data Innovation

    1 week ago

    Hubli, Karnataka, India beBeeDataLeader Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: Engineering ManagerAt the forefront of technological advancements, we are seeking a visionary leader to spearhead our Big Data team.The ideal candidate will possess in-depth expertise in Big Data platforms, Microsoft Azure, and Databricks. They will be responsible for overseeing a team of talented data engineers to design and deliver scalable data...

  • Big Data Solutions Developer

    1 day ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Data Engineer PositionJob Summary:We are seeking an experienced data engineer to design and implement large-scale data processing systems using Azure Databricks.Main Responsibilities:Develop and maintain high-quality data pipelines utilizing Azure Databricks, Python, and SQL.Collaborate with cross-functional teams to identify business requirements and create...