Senior Data Engineering Specialist

1 week ago

Gandhinagar, Gujarat, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

Job Opportunity: Data Engineering Professional

Overview of the Role:

We are seeking a seasoned data engineering professional with expertise in designing, developing, and optimizing large-scale data processing solutions. The ideal candidate will possess hands-on experience with PySpark, Python, SQL, and real-time data streaming, as well as contribute to the design and implementation of scalable data pipelines in cloud environments.

Key Responsibilities:

  • Data Pipeline Design and Development: Design, develop, and optimize data pipelines using PySpark, Spark, and Python.
  • ETL/ELT Solutions Integration: Implement ETL/ELT solutions to integrate structured and unstructured data from multiple sources.
  • Hive Queries and Data Transformations: Develop and maintain Hive queries and ensure seamless data transformations.
  • SQL Query Optimization: Write and optimize SQL queries for large datasets.
  • Real-Time Streaming Solutions: Build, monitor, and support real-time streaming solutions using Apache Kafka.
  • Databricks Management: Work on Databricks to create, manage, and optimize data workflows.
  • Azure Cloud Services Deployment: Deploy and manage solutions in Azure cloud services (e.g., Data Lake, Synapse, Azure Databricks).
  • Code Debugging and Optimization: Debug, troubleshoot, and optimize code for performance and scalability.
  • Collaboration and Communication: Collaborate with cross-functional teams to deliver data-driven solutions.

Required Skills and Qualifications:

  • Data Engineering Experience: 3-4 years of experience in data engineering with a focus on PySpark and Python.
  • Technical Knowledge: Strong knowledge of Spark, Hive, SQL, and Unix shell scripting.
  • Databricks Experience: Experience with Databricks in a production environment.
  • Azure Cloud Services: Hands-on experience with Azure cloud services (preferred: Data Lake, Synapse, Databricks).
  • Real-Time Streaming Technologies: Experience working with Apache Kafka or other real-time streaming technologies.
  • Problem-Solving and Debugging: Strong problem-solving and debugging skills.
  • Communication Skills: Excellent written and verbal communication skills.

Nice-to-Have Skills:

  • CI/CD Pipelines: Exposure to CI/CD pipelines for data solutions.
  • Data Governance and Security: Knowledge of data governance and security best practices.
  • Agile Methodology: Experience with Agile/Scrum methodology.

  • Senior Data Engineering Specialist

    1 week ago

    Gandhinagar, Gujarat, India beBeeSenior Full time ₹ 20,00,000 - ₹ 30,00,000

    We are seeking an experienced Senior ETL Developer to join our organization. The ideal candidate will have a proven track record of designing, developing, and deploying scalable data pipelines and ETL processes using Databricks.Key Responsibilities:Design and develop efficient ETL processes and data pipelines that meet business requirementsOptimize Spark...

  • IoT Data Engineer Specialist

    1 week ago

    Gandhinagar, Gujarat, India beBeeIoT Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: IoT Data Engineer Specialist">We are seeking a highly skilled IoT Data Engineer to join our team.Key Responsibilities:Develop and maintain data engineering solutions leveraging AWS IoT services, utilizing Python programming to build, optimize, and automate data pipelines.Work with large-scale IoT data ingestion, processing, and storage...

  • Lead Data Engineering Specialist

    2 weeks ago

    Gandhinagar, Gujarat, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Opportunity: Senior Data EngineerWe are seeking an experienced Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing and implementing large-scale data systems.Main Responsibilities:Design and implement scalable data pipelines using Azure and AWS Databricks.Develop and maintain high-quality data processing...

  • Lead Data Engineering Specialist

    1 week ago

    Gandhinagar, Gujarat, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer Job DescriptionWe are seeking a highly skilled Senior Data Engineer to lead our data engineering efforts. The ideal candidate will have expertise in cloud platforms, ETL processes, data warehousing, and big data technologies.Job RequirementsTo design, deploy, and maintain large-scale data platforms with high efficiency and scalability.To...

  • Senior Data Engineering Specialist

    7 days ago

    Gandhinagar, Gujarat, India beBeeBackendEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    As a Senior Backend Data & Integration Engineer, you will be responsible for designing and implementing crawling/fetch pipelines using API-first approach with playwright/requests. You will also work on parsing/normalization of job postings & CVs, deduplication/delta logic, embeddings/similarity search, integrations with HR4YOU, SerpAPI, BA job board,...

  • Expert Data Integration Specialist

    1 week ago

    Gandhinagar, Gujarat, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Senior Data Integration SpecialistWe are seeking a seasoned Senior Data Integration Specialist to lead our Informatica to SSIS migration projects. Key Responsibilities:Project Lead: Direct and execute Informatica to SSIS migration projects, ensuring accuracy, scalability, and performance.Design and develop robust ETL processes for data integration and...

  • Financial Data Specialist

    5 days ago

    Gandhinagar, Gujarat, India beBeeFinancial Full time ₹ 15,00,000 - ₹ 25,00,000

    Financial Data SpecialistWe are seeking a highly skilled Financial Data Specialist to handle day-to-day accounting tasks and support senior team members with financial activities.Key Responsibilities:Maintain accurate journal entries and financial recordsAssist in preparing reports and reconciliationsRequirements:1 year of experience in accounting or...

  • Senior Data Engineer Position

    1 week ago

    Gandhinagar, Gujarat, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    Job Description:As a senior data engineer, you will be responsible for designing and developing large-scale data pipelines using Azure Data Factory and Databricks. This role involves collaborating with cross-functional teams to identify business requirements and develop data-driven solutions.The ideal candidate will have a strong understanding of data...

  • Senior Data Engineer

    7 days ago

    Gandhinagar, Gujarat, India beBeeDataEngineering Full time ₹ 26,81,351 - ₹ 31,35,817

    Our Ideal CandidateWe are seeking an accomplished Data Engineering Leader to spearhead the development of our self-service data platform. The ideal candidate will have a strong background in designing scalable data solutions, leveraging technologies such as Databricks, SQL, Python, Spark, and Delta Lake on Azure.Key Responsibilities:Design and implement data...

  • Senior Cloud Data Specialist

    1 week ago

    Gandhinagar, Gujarat, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Data Engineer Position">As a senior data engineer, you will be responsible for designing and developing large-scale data processing systems. Your expertise in Python and cloud computing will be crucial in driving business success.Responsibilities:Design, develop, and maintain scalable data infrastructure solutionsCollaborate with cross-functional teams to...