Senior Data Engineer Role

3 days ago

Gandhinagar, Gujarat, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

Job Description

  • As a senior data engineer, you will be responsible for analyzing existing technology landscapes to understand current data integration frameworks and their impact on project requirements.
  • You will develop Spark jobs using Scala for new project requirements and enhance existing Spark jobs for ongoing product enhancements.
  • Performance tuning of Spark jobs, including stress testing, is also part of your responsibilities.
  • Additionally, you will create new data pipelines for developed/enhanced Spark jobs using AWS Lambda or Apache Airflow.
  • You will lead the database design process, covering logical design, physical design, star schema, and snowflake schema.
  • Another key aspect of this role is analyzing data processing, integration, modeling, and reporting requirements, and defining data loading strategies considering volume, data types, frequency, and analytics specifications.
  • Ensuring an optimal balance between cost and performance is crucial in this position.
  • You will prepare project documentation, adhere to quality guidelines, and meet schedules.
  • Closely collaborating with Project Managers for successful project delivery, providing estimation, scoping, and scheduling assistance is essential.
  • Managing the build phase and ensuring code quality to meet requirements and adhere to cloud architecture are also key responsibilities.

Required Skills and Qualifications

  • A minimum of 7-9 years of IT experience
  • At least 5+ years of AWS-related projects
  • Associate Level and Professional Level AWS Certification (good to have)
  • In-depth knowledge of AWS Services: S3, EC2, EMR, Serverless, Athena, AWS Glue, Lambda, Step Functions
  • Cloud Databases: AWS Aurora, Singlestore, RedShift, Snowflake
  • Big Data: Hadoop, Hive, Spark, YARN
  • Programming Language: Scala, Python, Shell Scripts, PySpark
  • Operating System: Any flavor of Linux, Windows
  • Strong SQL Skills
  • Orchestration Tools: Apache Airflow
  • Expertise in developing ETL workflows with complex transformations like SCD, deduplications, aggregations etc.
  • Thorough understanding of AWS VPC, Subnets, Security Groups, and Route Tables
  • Ability to learn quickly and adapt to new AWS Services or Big Data Technologies as required

Benefits

  • Bachelor's degree in Computer Science, Engineering, or a related field (Master's degree is a plus)
  • Continued learning through technical certifications or related methods
  • Minimum 5 years of experience on Cloud-related projects

Others

  • Hold strong technical knowledge and experience
  • Ability to deep dive and research in various technical fields
  • Self-motivated and focused on delivering outcomes for a fast-growing team and firm
  • Able to communicate persuasively through speaking, writing, and client presentations
  • Able to consult, write, and present persuasively
  • Able to work in a self-organized and cross-functional team
  • Able to iterate based on new information, peer reviews, and feedback
  • Prior experience working in a large media company would be an added advantage

  • Senior Data Engineer Role

    16 hours ago

    Gandhinagar, Gujarat, India beBeeDataEngineering Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job Title: Data Engineering LeadWe are seeking an experienced data engineering lead to spearhead our data initiatives. This is a leadership role that requires strong technical expertise in designing, building, and maintaining large-scale data systems.Key Responsibilities:

  • Senior Data Engineer Role

    3 days ago

    Gandhinagar, Gujarat, India beBeeDataEngineer Full time ₹ 13,50,000 - ₹ 20,25,000

    Job DescriptionWe are seeking a skilled Azure & Data ETL Developer to join our dynamic data engineering team. As an integral part of this team, you will play a pivotal role in designing and implementing data processing solutions using Azure Functions (.NET). Your expertise will be instrumental in building robust ELT pipelines with ADF / Azure Data Services,...

  • Senior Data Engineer

    7 days ago

    Gandhinagar, Gujarat, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job TitleA senior data engineer is required to develop and optimize ETL/ELT pipelines using Azure Data Factory (ADF), Databricks, and Synapse Analytics.This role involves designing and implementing scalable data architectures following Medallion Architecture and best practices.The ideal candidate will have experience in Azure Data Engineering with Data...

  • Senior Data Engineer Position

    16 hours ago

    Gandhinagar, Gujarat, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    Job Description:As a senior data engineer, you will be responsible for designing and developing large-scale data pipelines using Azure Data Factory and Databricks. This role involves collaborating with cross-functional teams to identify business requirements and develop data-driven solutions.The ideal candidate will have a strong understanding of data...

  • Senior Cloud Data Engineer

    2 weeks ago

    Gandhinagar, Gujarat, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,40,00,000

    Job Opportunity Overview">We are seeking highly skilled professionals to join a mission-critical banking project as Senior Data Engineers.This is your chance to work on cutting-edge data pipelines, streaming platforms, and cloud-based infrastructures—building scalable, high-performance solutions for a leading financial services environment.What You'll Work...

  • Senior Data Engineer

    5 days ago

    Gandhinagar, Gujarat, India The Glove Full time

    Job DescriptionAbout the Company:We are a leading organization in the tech industry, committed to delivering innovative solutions that drive business transformation. Our team is passionate about leveraging cutting-edge technologies to solve complex challenges and create value for our clients. We foster a collaborative and inclusive work environment that...

  • Data Engineer Role

    3 days ago

    Gandhinagar, Gujarat, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job Title: Data Engineer with FabricWe are seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will play a critical role in designing, developing, and maintaining our data pipelines and infrastructure.Key Responsibilities:Design, develop, and implement data pipelines using Azure Data Factory and Databricks to support the...

  • Senior Data Engineering Specialist

    14 hours ago

    Gandhinagar, Gujarat, India beBeeSenior Full time ₹ 20,00,000 - ₹ 30,00,000

    We are seeking an experienced Senior ETL Developer to join our organization. The ideal candidate will have a proven track record of designing, developing, and deploying scalable data pipelines and ETL processes using Databricks.Key Responsibilities:Design and develop efficient ETL processes and data pipelines that meet business requirementsOptimize Spark...

  • Lead Data Engineering Specialist

    14 hours ago

    Gandhinagar, Gujarat, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer Job DescriptionWe are seeking a highly skilled Senior Data Engineer to lead our data engineering efforts. The ideal candidate will have expertise in cloud platforms, ETL processes, data warehousing, and big data technologies.Job RequirementsTo design, deploy, and maintain large-scale data platforms with high efficiency and scalability.To...

  • Senior Test Engineer

    1 day ago

    Gandhinagar, Gujarat, India Molecular Connections Full time

    Company Description Molecular Connections (MC) is a leading provider of Big Data and Data Science Solutions, using AI-powered models to drive digital transformation for customers. With a focus on innovation, MC collaborates with pharma and STEM industries to offer end-to-end software development and data insights. The company prides itself on empowering...