Big Data Pipelines Specialist

2 days ago

Allahabad, Uttar Pradesh, India beBeeSnowflake Full time ₹ 20,00,000 - ₹ 25,00,000
Snowflake Data Engineer

We're looking for a skilled data engineer to design and implement data pipelines using Snowflake Tasks, Streams, and Stored Procedures.

  • Design and build ETL/ELT pipelines using Snowflake Tasks, Streams, and Stored Procedures to efficiently manage and process large datasets.
  • Develop data models with clustering keys and optimize micro-partitioning to improve query performance and reduce storage costs.
  • Implement real-time ingestion and external tables for data lake integration to ensure seamless data flow.
  • Develop JavaScript/SQL stored procedures and UDFs for complex transformations using window functions, CTEs, and pivots.
  • Write complex SQL queries to optimize query performance and manage multi-cluster warehouses to control costs.

Requirements:

  • A strong understanding of Snowflake development and a solid grasp of Snowflake SQL and basic JavaScript stored procedures.
  • Working knowledge of Tasks, Streams, Stages, File Formats, Time Travel, and RBAC.
  • Experience with dbt core concepts (models, sources, tests, snapshots) and warehouse sizing, auto-suspend/resume, and query optimization.
  • Familiarity with semi-structured data (JSON/XML) and the SnowSQL CLI and Snowflake web interface.

  • Big Data Analytics Specialist

    2 days ago

    Allahabad, Uttar Pradesh, India beBeeAnalytics Full time ₹ 8,00,000 - ₹ 16,00,000

    Job Title: Big Data Hadoop Scala EngineerWe are seeking an experienced Big Data Analytics Specialist to join our team. The ideal candidate will have hands-on experience with Scala, Spark, Hive, and Hadoop Big Data Eco System Tools.Key Responsibilities:Data Analysis and Optimization: Develop and optimize complex queries using SQL, focusing on performance...

  • Data Pipeline Specialist

    1 week ago

    Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 12,00,000

    Job Overview:We are seeking an accomplished Data Engineer to spearhead our data pipeline initiatives.About the Role:The ideal candidate will possess experience in designing and implementing data pipelines utilizing Google Cloud Platform and Azure Databricks. Proficiency in DBT and a solid understanding of secure coding practices are essential.In addition,...

  • Data Engineering Specialist

    5 days ago

    Allahabad, Uttar Pradesh, India beBeeBigData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: Data Engineering SpecialistJob Overview:As a Data Engineering Specialist, you will play a pivotal role in designing and developing large-scale data processing systems using cutting-edge technologies.This challenging position requires expertise in big data technologies, including PySpark and Hive. Familiarity with Spark, HBase, and data quality...

  • Freelance Big Data Specialist

    5 days ago

    Allahabad, Uttar Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job TitleWe are seeking a highly skilled Freelance Data Engineer with expertise in designing, building, and optimizing large-scale data pipelines using Python, SQL, and cloud-based technologies.Data Engineering Expertise: Develop and maintain ETL/ELT pipelines using PySpark, AWS, and Databricks to extract, transform, and load data from various sources.Big...

  • Big Data Analyst

    2 days ago

    Allahabad, Uttar Pradesh, India beBeeData Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Job Title: Data Analyst SpecialistWe are seeking a skilled data analyst to join our team.Overview:The position is ideal for professionals with 6–12 years of experience in big data and related technologies. As a data analyst, you will be responsible for analyzing and interpreting large datasets to gain insights and make informed...

  • Senior Data Pipeline Developer

    2 weeks ago

    Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Highly skilled data professionals are sought after to spearhead the development of complex data pipelines. This role involves leveraging expertise in streaming data environments and frameworks such as Apache Spark, Streamsets, or Apache NIFI to ensure high-quality data delivery.Mandatory Skills:Experience with Kafka Framework, including kSQL and Mirror...

  • Lead Big Data Engineer

    1 week ago

    Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job DescriptionWe are seeking a senior data engineer to lead our big data engineering team. As a key member of the team, you will be responsible for developing architectures that meet business needs and providing technical leadership.The role involves setting up cloud-based data lakes and implementing standardized data models. You will develop data pipelines...

  • Big Data Solutions Specialist

    6 days ago

    Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Big Data EngineerWe are seeking a highly skilled professional with expertise in Scala, Spark, Python, NiFi, and Apache Kafka to join our data engineering team.Key Responsibilities:Design, develop, and optimize large-scale data pipelines and distributed data processing systems.Work extensively with Scala, Spark (PySpark), and Python for data...

  • Senior Big Data Specialist

    2 days ago

    Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Data Engineering RoleWe are seeking a skilled professional to fill the position of Data Engineer.Responsibilities:Design and develop large-scale data systems.Optimize existing projects for improved performance.Research and implement Big Data tools and frameworks.Develop scalable solutions that meet business/data requirements.Create reusable components and...

  • Data Pipeline Specialist

    6 days ago

    Allahabad, Uttar Pradesh, India beBeeSenior Full time ₹ 12,00,000 - ₹ 30,00,000

    Job Title: Senior Test Engineer">The role of a senior test engineer is multifaceted, requiring expertise in designing and executing manual and automated test cases for complex data pipelines.">Key Responsibilities:">">Design and develop test cases for Azure Data Factory ETL pipelines using Python and Pytest.">Validate data transformations, mappings, and...