Big Data Solutions Architect

2 weeks ago

Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

About the Role

A highly skilled Big Data Engineer is required to lead a team of big data engineers. This individual will work closely with clients and team members to understand their requirements and develop architectures that meet their needs.

Key Responsibilities:

  • Support the Data Engineering team in setting up the Data Lake on Cloud and implementing standardized Data Model, single view of customer.
  • Develop data pipelines for new sources, data transformations within the Data Lake, implementing GraphQL, working on No SQL database, CI/CD and data delivery as per business requirements.
  • Build pipelines to bring in wide variety of data from multiple sources within the organization as well as from social media and public data sources.
  • Collaborate with cross-functional teams to source data and make it available for downstream consumption.
  • Work with the team to provide an effective solution design to meet business needs.

Requirements:

  • Experience with at least one Cloud Infra provider (Azure/AWS).
  • Experience in building data pipelines using batch processing with Apache Spark (Spark SQL, Dataframe API) or Hive query language (HQL).
  • Experience in building streaming data pipeline using Apache Spark Structured Streaming or Apache Flink on Kafka & Delta Lake.
  • Knowledge of NoSQL databases, good to have experience in Cosmos DB, Restful APIs and GraphQL.
  • Knowledge of Big data ETL processing tools, Data modelling and Data mapping.
  • Experience with Hive and Hadoop file formats (Avro / Parquet / ORC).
  • Basic knowledge of scripting (shell / bash).
  • Experience of working with multiple data sources including relational databases (SQL Server / Oracle / DB2 / Netezza), NoSQL / document databases, flat files.
  • Basic understanding of CI/CD tools such as Jenkins, JIRA, Bitbucket, Artifactory, Bamboo and Azure Dev-ops.
  • Basic understanding of DevOps practices using Git version control.
  • Ability to debug, fine-tune and optimize large-scale data processing jobs.

Working Conditions

This role requires strong analytical skills and excellent communication skills. The ideal candidate should be able to work effectively in a fast-paced environment and collaborate with diverse teams.

About Us

  • Big Data Specialist

    2 hours ago

    Hubli, Karnataka, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Big Data Engineer - Job OpportunityWe are seeking a highly skilled professional with expertise in Big Data technologies and AWS Cloud Services. The ideal candidate will bring strong hands-on experience in designing, architecting, and implementing scalable data engineering solutions while driving innovation within the team.Design, develop, and optimize Big...

  • Senior Big Data Architect

    13 hours ago

    Hubli, Karnataka, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Big Data Engineer PositionJob Summary:We are seeking a highly skilled Big Data Engineer to design and implement scalable data transformation pipelines using Snowflake and dbt, following industry-accepted best practices.Key Responsibilities:Data Pipeline Development: Design and implement robust ELT pipelines using dbt on Snowflake, ensuring scalability and...

  • Senior Big Data Architect

    3 days ago

    Hubli, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,20,00,000

    Big Data Systems DeveloperWe are seeking a skilled Data Engineer II to join our team. This role is crucial in designing and executing large-scale distributed data systems.The ideal candidate should have 2 to 4 years of experience in big data technologies, such as Apache Hadoop, and relational databases, including MS SQL Server, Oracle, MySQL, and...

  • Senior Data Solutions Architect

    1 week ago

    Hubli, Karnataka, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Description:Data Engineering LeadershipWe are seeking a seasoned Data Solutions Architect to lead the design, development, and implementation of scalable, secure, and high-performance data solutions on Databricks.The ideal candidate will have a strong background in data engineering, with expertise in Databricks, PySpark, Spark SQL, and Scala. They will...

  • Big Data Solutions Developer

    15 hours ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Data Engineer PositionJob Summary:We are seeking an experienced data engineer to design and implement large-scale data processing systems using Azure Databricks.Main Responsibilities:Develop and maintain high-quality data pipelines utilizing Azure Databricks, Python, and SQL.Collaborate with cross-functional teams to identify business requirements and create...

  • Cloud Data Architect

    55 minutes ago

    Hubli, Karnataka, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job Title: Cloud Data ArchitectAbout the Role:As a Cloud Data Architect, you will be responsible for designing and implementing scalable data solutions on cloud platforms.Design and implement data pipelines, warehouses, and data lakes.Work on data modeling, ETL/ELT processes, and performance tuning.Collaborate with business and technical teams to deliver...

  • Cloud Architect

    3 days ago

    Hubli, Karnataka, India beBeeDataScientist Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Description:We are looking for skilled and passionate experts to join our team at Bengaluru. This is a full-time hybrid role offering flexibility for remote work.Key Responsibilities:Design Big Data architecture, cloud adoption strategies, and solutions.Migrate on-premises applications and databases to the cloud.Ensure data governance and applicative...

  • Data Solutions Architect

    1 day ago

    Hubli, Karnataka, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    About People Prime Worldwide: We provide digital engineering and technology services to top companies worldwide. Our Engineering First approach ensures we can execute all ideas and creatively solve business challenges. With expertise and empowered agile teams, we prioritize execution for impactful results. We combine logic, creativity and curiosity to build,...

  • Big Data Specialist

    3 days ago

    Hubli, Karnataka, India beBeeDataScientist Full time ₹ 60,00,000 - ₹ 1,20,00,000

    Job OverviewThe role involves working with large-scale data sets utilizing Big Data tools and technologies. We are looking for a skilled professional to join our team as a Data Scientist.Key Responsibilities:Develop, tweak queries, and work on performance enhancement using Scala, Spark, Hive, and Hadoop Big Data Eco System Tools.Apply SQL experience to...

  • Chief Data Architect

    13 hours ago

    Hubli, Karnataka, India beBeeDataLeader Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job TitleWe are seeking a skilled data leader with expertise in SQL, big data processing, and modern data platforms. The ideal candidate will have hands-on experience with modern unified data platforms and exposure to Microsoft Azure Fabric.Key Responsibilities:Develop and continuously improve the Data/AI PracticeDesign, build, and optimise data pipelines...