Big Data Developer

1 day ago

Amrāvati, Maharashtra, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 2,20,00,000
Job Title: Data Engineer

We are seeking an experienced Databricks PySpark Developer to join our data platform implementation team. In this role, you will design, develop, and maintain efficient ETL processes to extract, transform, and load data into our data lake and data warehouse.

Key Responsibilities:
  • Design and develop ETL processes using Databricks PySpark to extract, transform, and load data from various sources into our data lake and data warehouse.
  • Optimize ETL workflows for performance and scalability, leveraging Databricks PySpark and Spark SQL to efficiently process large data volumes.
  • Implement robust error handling and monitoring mechanisms to proactively detect and resolve issues within ETL processes.
  • Design and implement data solutions following the Medallion Architecture principles, organizing data into Bronze, Silver, and Gold layers.
  • Ensure data is appropriately cleansed, enriched, and optimized at each stage to support robust analytics and reporting.
Requirements:
  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • Minimum of 3 years of experience as an ETL developer, with a strong focus on Databricks PySpark development.
  • Proficiency in Python programming, with extensive experience in developing and debugging Databricks PySpark applications.
  • In-depth understanding of Spark architecture and internals, with hands-on experience in Spark RDDs, DataFrames, and Spark SQL.
  • Expertise in writing and optimizing complex SQL queries for data manipulation, aggregation, and analysis.
  • Proven experience in working with large-scale data warehousing and ETL frameworks.
  • Strong problem-solving skills and the ability to troubleshoot and resolve ETL process issues.
  • Excellent communication and collaboration skills, with the ability to work effectively in a team environment.

  • Big Data Architect

    1 day ago

    Amrāvati, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    **Job Title:** Big Data ArchitectWe are seeking a highly skilled Big Data Architect to design, develop, and optimize large-scale data pipelines and distributed data processing systems.The ideal candidate will have a proven track record in building scalable cloud-based data architectures using Scala, Spark, Python, Apache Kafka, and NiFi.

  • Big Data Solutions Expert

    13 hours ago

    Amrāvati, Maharashtra, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Unlock your expertise in data processing and analytics as a Big Data Engineer.Key Responsibilities:Develop scalable big data solutions using PySpark and Hive to drive business growth.Collaborate with cross-functional teams to integrate big data platforms with other systems, ensuring seamless data flow.Work closely with data scientists and analysts to build...

  • Big Data Platform Visionary

    1 day ago

    Amrāvati, Maharashtra, India beBeeDataLeader Full time US$ 1,25,000 - US$ 1,75,000

    Job OverviewWe are seeking a visionary leader to spearhead the development of a cutting-edge big data platform.Align with key client initiatives to drive strategic growth and collaboration

  • Senior Big Data Specialist

    3 days ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 15,00,000

    Job Title: Data EngineerSynopsis:As a seasoned data engineer, you will be responsible for designing and developing large-scale data processing systems using distributed computing frameworks like PySpark.Our ideal candidate is an expert in Big Data technologies, with hands-on experience in Python, AWS, and Spark.Mandatory Skills:PythonPySparkAWSBonus...

  • Big Data Ingestion Specialist

    4 days ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Azure Databricks is a leading platform for big data analytics and machine learning. We are seeking an experienced professional to join our team and work on building scalable and efficient data ingestion pipelines.The ideal candidate will have hands-on experience with data pipelines, automation, and infrastructure management to support the integration of...

  • Senior Big Data Project Coordinator

    18 hours ago

    Amrāvati, Maharashtra, India beBeeProject Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Technical Project Manager PositionWe seek an accomplished Technical Project Manager to spearhead large-scale projects within complex big data and analytics environments.

  • Senior Big Data Solutions Architect Position

    2 days ago

    Amrāvati, Maharashtra, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,30,00,000

    Big Data Architect Job OpportunityThis role is for an experienced Big Data Architect to lead multiple data engagements on cloud platforms.The ideal candidate will have more than 8 years of experience in technical roles, with a strong background in architecting, designing, and implementing end-to-end data pipelines and data integration solutions for varied...

  • Big Data Architect

    3 days ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    We are seeking a highly skilled professional to build and maintain large-scale data infrastructures, enabling business growth through real-time insights.Key Responsibilities:Design, develop, and deploy scalable data pipelines using industry-leading technologies like Kafka, Flink, and Spark.Create robust data models ensuring high-quality and efficient data...

  • Cloud-Based Data Architect

    3 days ago

    Amrāvati, Maharashtra, India beBeeData Full time ₹ 10,00,000 - ₹ 23,00,000

    Job Title:Big Data EngineerAre you an experienced data engineer looking for a new challenge?We are seeking a skilled Big Data Engineer to join our team.Design and develop large-scale data processing systems using Azure Databricks and Pyspark.Collaborate with cross-functional teams to integrate data engineering solutions with business applications.Develop and...

  • Senior Big Data Architect

    2 days ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    We are seeking a highly skilled Data Engineer to design, develop and maintain ETL/ELT data pipelines.This role involves working closely with stakeholders to understand project requirements and translate them into technical solutions using AWS services such as AWS Lambda, AWS DMS and AWS Glue.As a Data Engineer your key responsibilities include:Designing...