Senior Cloud Data Engineer

1 day ago

Kollam, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000
Data Engineer Job Description

We are seeking a seasoned and proficient Senior Data Engineer with substantial experience in cloud technologies.

As a pivotal member of our data engineering team, you will play a crucial role in designing, implementing, and optimizing data pipelines, ensuring seamless integration with cloud platforms.

  • Key Responsibilities:

  • Design, develop, and maintain scalable and efficient data pipelines using Python and cloud-based technologies.
  • Implement Extract, Transform, Load (ETL) processes to seamlessly move data from diverse sources into our cloud-based data warehouse.
  • Utilize cloud platforms to deploy, manage, and optimize data engineering solutions.
  • Leverage cloud-native services for storage, processing, and analysis of large datasets.
  • Collaborate with data scientists, analysts, and other stakeholders to design effective data models that align with business requirements.
  • Ensure the scalability, reliability, and performance of the overall data infrastructure on cloud platforms.
  • Continuously optimize data processes for improved performance, scalability, and cost-effectiveness in a cloud environment.
  • Monitor and troubleshoot issues, ensuring timely resolution and minimal impact on data availability.
  • Implement data quality checks and validation processes to ensure the accuracy and completeness of data in the cloud-based data warehouse.
  • Collaborate with cross-functional teams to identify and address data quality issues.

Required Skills and Qualifications:

  • Programming Languages: Proficiency in Python for data engineering tasks, scripting, and automation.
  • Data Engineering Technologies: Extensive experience with data engineering frameworks like distributed data processing.
  • Understanding and hands-on experience with workflow management tools like Apache Airflow.
  • Cloud Platforms: In-depth knowledge and hands-on experience with at least one major cloud platform.
  • Familiarity with cloud-native services for data processing, storage, and analytics.
  • ETL Processes: Proven expertise in designing and implementing Extract, Transform, Load (ETL) processes.
  • SQL and Databases: Proficient in SQL with experience in working with relational databases and cloud-based database services.
  • Data Modeling: Strong understanding of data modeling principles and experience in designing effective data models.
  • Version Control: Familiarity with version control systems, such as Git, for tracking changes in code and configurations.
  • Collaboration Tools: Experience using collaboration and project management tools for effective communication and project tracking.
  • Containerization and Orchestration: Familiarity with containerization technologies and orchestration tools.
  • Monitoring and Troubleshooting: Ability to implement monitoring solutions and troubleshoot issues in data pipelines.
  • Data Quality Assurance: Experience in implementing data quality checks and validation processes.
  • Agile Methodologies: Familiarity with agile development methodologies and practices.

Benefits:

  • Opportunity to work with a talented team of data engineers and scientists.
  • Challenging projects that require innovative solutions.
  • Professional growth and development opportunities.

Others:

  • Strong problem-solving and critical-thinking abilities.
  • Excellent communication skills, both written and verbal.
  • Ability to work collaboratively in a cross-functional team environment.
  • Attention to detail and commitment to delivering high-quality solutions.

  • Cloud-Based Data Engineer Lead

    18 hours ago

    Kollam, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Lead Data Engineer PositionJob Summary:This senior-level position is available for a highly skilled and experienced data engineer who can lead the development and implementation of scalable data pipelines using cloud-based technologies.Responsibilities:Design, develop, and optimize ETL/ELT data pipelines using Apache Spark (PySpark or Scala), AWS Glue, and...

  • Senior Cloud Data Specialist

    2 days ago

    Kollam, Kerala, India beBeeDatabricksspecialist Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Overview:We are seeking a seasoned professional to serve as a Senior Databricks Data Engineer on our data engineering team. This is an exciting opportunity for the right candidate to utilize their skills in PySpark, CI/CD pipelines, and Terraform for infrastructure as code to drive success in our organization.Design, develop, and maintain scalable and...

  • Cloud Data Specialist

    2 days ago

    Kollam, Kerala, India beBeeDataEngineer Full time ₹ 60,00,000 - ₹ 80,00,000

    Senior Cloud Data EngineerWe are seeking a seasoned Senior Cloud Data Engineer to join our team. As a key member of our data engineering group, you will be responsible for designing, developing, and deploying large-scale data pipelines on cloud infrastructure using services such as Amazon S3, AWS Glue, AWS Lambda, Amazon Redshift, and others.The ideal...

  • Data Scientist/ML Engineer

    4 days ago

    Kollam, Kerala, India Quant-data Full time

    We're Hiring: Machine Learning Engineer / Data Engineer (Remote | Full-Time) Build AI-powered credit decisioning systems on Microsoft AzureWe're looking for a Machine Learning Engineer / Data Engineer with 5+ years of experience to join our AI-driven credit lending platform team. In this role, you'll design and deploy scalable ML solutions that power loan...

  • Cloud Data Engineer

    23 hours ago

    Kollam, Kerala, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    We are seeking an experienced Data Engineer to spearhead our cloud-based data operations.Key Responsibilities:Data Pipeline DevelopmentDesign and develop scalable ETL pipelines using Azure Data Factory, Databricks, and Spark for batch data ingestion and egress.Data Transformation WorkflowsImplement data transformation workflows using PySpark and Scala Spark...

  • Data Engineering Cloud Specialist

    18 hours ago

    Kollam, Kerala, India beBeeCloudEngineer Full time ₹ 80,00,000 - ₹ 2,00,00,000

    Cloud Data Engineer Role Overview">We are looking for a highly skilled professional to join our team as a Cloud Data Engineer. This is an excellent opportunity to utilize your expertise in cloud-based technologies, specifically Google Cloud Platform (GCP), and work on designing, developing, and maintaining large-scale data processing systems.Key...

  • Data engineer

    2 days ago

    Kollam, Kerala, India Centrilogic Full time

    Data EngineerPurpose:Over 15 years, we have become a premier global provider of multi-cloud management, cloud-native application development solutions, and strategic end-to-end digital transformation services.Headquartered in Canada and with regional headquarters in the U. S. and the United Kingdom, Centrilogic delivers smart, streamlined solutions to...

  • Data Engineer for Cloud Transformation

    4 days ago

    Kollam, Kerala, India beBeeCloudTransformation Full time ₹ 18,00,000 - ₹ 20,00,000

    We are seeking a seasoned data engineer to lead cloud transformation initiatives, focusing on full lifecycle implementations from secure platform setup and cloud integration to high-performance data pipeline development.This role requires expertise in Snowflake, Terraform or Pulumi, CI/CD pipelines, ETL/ELT pipelines using dbt or Airflow, and integrating...

  • Senior Cloud Data Architect

    2 days ago

    Kollam, Kerala, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Senior Software Engineer/Technical SpecialistJob DescriptionWe are seeking a skilled software engineer to join our team. As a senior software engineer, you will be responsible for designing and implementing scalable data architectures using Azure Data Factory (ADF), Databricks, and Synapse Analytics.You will work with large datasets, developing...

  • Senior Data Engineer

    18 hours ago

    Kollam, Kerala, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Senior Data EngineerThis senior data engineer position offers the opportunity to develop and maintain scalable data pipelines using Python and AWS services (S3, Redshift, Lambda, Glue) ensuring efficient data processing and transfer.The selected candidate will be responsible for implementing and optimizing Extract, Transform, Load (ETL) processes...