Current jobs related to Big Data Engineer - Cochin, Kerala - beBeeDataArchitect

  • Data Engineer Big Data

    2 weeks ago

    Cochin, Kerala, India Litmus7 Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Design, develop, and maintain scalable and efficient big data processing pipelines distributed computing systems.Collaborate with cross-functional teams to understand data requirements and design appropriate data solutions.Implement data ingestion, processing, and transformation processes to support various analytical and machine learning use cases.Optimize...

  • Senior Big Data Administrator

    1 week ago

    Cochin, Kerala, India beBeeDataiku Full time ₹ 10,00,000 - ₹ 23,45,678

    Big Data Administrator Job Description">We are seeking a highly skilled Big Data Administrator to manage our big data platform. This is an exciting opportunity for experienced professionals to work with cutting-edge technology and contribute to the success of our organization.">The ideal candidate will have extensive knowledge of Hadoop, Hive, HBase, and...

  • Experienced Big Data Architect

    1 week ago

    Cochin, Kerala, India beBeeCloud Full time ₹ 15,00,000 - ₹ 20,10,000

    Job Title: Experienced Big Data ArchitectWe are seeking a highly skilled individual to join our team as a Big Data Architect.Key Responsibilities:Design and develop scalable data engineering solutions leveraging AWS services for large-scale, complex data processing.Build and maintain data pipelines using Spark (Scala) for both structured and unstructured...

  • Big Data Systems Architect

    2 weeks ago

    Cochin, Kerala, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 1,50,00,000

    Role SummaryData Engineering is a highly skilled and specialized field that involves designing, building, and maintaining large-scale data systems.We are seeking an experienced Data Engineer to join our team. As a Data Engineer at our company, you will be responsible for the design, execution, and management of distributed data systems, including monitoring...

  • Senior Big Data Expert

    2 weeks ago

    Cochin, Kerala, India beBeeDataengineer Full time ₹ 15,00,000 - ₹ 28,00,000

    Data Engineer PositionWe are seeking an experienced Data Engineer to lead our data engineering efforts. Job Description:As a Senior Big Data Expert, you will be responsible for designing and implementing large-scale data processing systems using technologies such as Apache Spark, Hadoop, and cloud-based platforms like Azure, AWS, or GCP.Required Skills and...

  • Unlock the Power of Big Data: Senior Data Engineer Wanted

    1 week ago

    Cochin, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Data EngineerThis role involves designing, developing and maintaining large-scale data systems. We are seeking an experienced professional who can lead our data engineering efforts.Key Responsibilities:Developing and implementing data pipelines using PythonCollaborating with cross-functional teams to integrate data systems with cloud platforms,...

  • Data Engineer

    1 week ago

    Cochin, Kerala, India Prevalent AI Full time ₹ 1,50,000 - ₹ 28,00,000 per year

    Role purpose Are you passionate about cutting-edge technology and big data? At Prevalent AI, we're at the forefront of building innovative solutions to handle petabyte-scale data. As a Data Engineer, you will play a pivotal role in supporting the development of key data engineering components for our Data Fabric and Exposure Management products.In this...

  • High-Powered Big Data Specialist

    2 weeks ago

    Cochin, Kerala, India beBeeDataEngineer Full time ₹ 30,00,000 - ₹ 50,00,000

    Job Title: Data EngineerWe are looking for a skilled Data Engineer to join our team. As a key member of our organization, you will play a vital role in the design, development, and maintenance of our data infrastructure.Responsibilities:Design, build, and deploy scalable data processing systems using big data technologies such as AWS, Python, and...

  • Big Data Analytics Expert

    2 weeks ago

    Cochin, Kerala, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OverviewThe Role of a Big Data DeveloperThis is an exciting opportunity to work with big data, leveraging technologies like Scala, AWS, and Python or Java. ResponsibilitiesAnalyzing large datasets to gain valuable insightsDeveloping and maintaining scalable data processing systemsEnsuring data quality and integrityMust have experience in handling big...

  • Data Engineer Position

    2 weeks ago

    Cochin, Kerala, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 35,00,000

    Job Title: Data ArchitectWe are looking for a seasoned professional with expertise in data engineering to join our team.Job DescriptionWe are seeking an experienced data engineer to design and implement solutions in the areas of data warehousing, big data, and data analytics. The ideal candidate will have strong knowledge of Google Cloud Platform (GCP) data...

Big Data Engineer

2 weeks ago

Cochin, Kerala, India beBeeDataArchitect Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

We are looking for a skilled Data Architect to join our data platform implementation team.

The successful candidate will be responsible for designing and developing efficient ETL processes using Databricks PySpark to extract, transform, and load data from various sources into our data lake and data warehouse.

Key Responsibilities:

Data Development

  • Design and develop ETL processes to extract, transform, and load data from heterogeneous sources.
  • Optimize ETL workflows for performance and scalability.
  • Implement robust error handling and monitoring mechanisms to proactively detect and resolve issues within ETL processes.
  • Organize data into Bronze, Silver, and Gold layers following the Medallion Architecture principles.
  • Cleanse, enrich, and optimize data at each stage to support robust analytics and reporting.

Data Pipeline Management

  • Develop and maintain data pipelines using Databricks PySpark.
  • Collaborate with data engineering teams to translate data requirements into efficient ETL workflows and pipelines.

Data Analysis and Query Optimization

  • Write and optimize complex SQL queries for data manipulation, aggregation, and analysis.

Project Coordination and Continuous Improvement

  • Participate in project planning activities to ensure timely delivery of ETL solutions.
  • Stay updated on the latest developments in Databricks PySpark and related technologies.
  • Document ETL processes to facilitate knowledge sharing and ensure compliance with data governance standards.

Requirements

Bachelor's degree in Computer Science or a related field is required.

Minimum 3 years of experience as an ETL developer is necessary. Proficiency in Python programming and extensive experience in developing Databricks PySpark applications are essential.

In-depth understanding of Spark architecture and internals, hands-on experience in Spark RDDs, DataFrames, and Spark SQL, and expertise in writing complex SQL queries are required.

Strong problem-solving skills and the ability to troubleshoot and resolve ETL process issues are mandatory.

Preferred Qualifications

Experience with cloud platforms and data platform tools such as DataBricks, Snowflake, and Tableau is preferred.

],