Senior Big Data Developer

2 days ago

Kota, Rajasthan, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

Job Title: Data Engineer

About the Role

We're seeking a skilled Data Engineer to create and maintain our data infrastructure. In this position, you'll be responsible for designing and developing ETL processes to extract, transform, and load data from various sources into our data lake and warehouse.

Main Responsibilities:
  • ETL Development
    • Design and develop ETL processes using Databricks PySpark to extract, transform, and load data from diverse sources into our data lake and warehouse.
    • Optimize ETL workflows for performance and scalability, leveraging Databricks PySpark and Spark SQL to efficiently process large data volumes.
    • Implement robust error handling and monitoring mechanisms to proactively detect and resolve issues within ETL processes.
    • Create data solutions following Medallion Architecture principles, organizing data into Bronze, Silver, and Gold layers.
    • Evaluate data quality, integrity, and reliability throughout the ETL lifecycle.
  • Data Pipeline Management
    • Develop advanced data pipelines using Databricks workflows.
    • Maintain data pipelines using Databricks PySpark, ensuring data quality, integrity, and reliability throughout the ETL lifecycle.
    • Collaborate with data engineering, science, and business intelligence teams to translate data requirements into efficient ETL workflows and pipelines.
  • Data Analysis and Query Optimization
    • Write and optimize complex SQL queries for data manipulation, aggregation, and analysis within Databricks PySpark applications.
  • Project Coordination and Continuous Improvement
    • Participate in project planning and coordination activities to ensure timely delivery of ETL solutions.
    • Stay updated on Databricks PySpark, Spark SQL, and related technologies, recommending and implementing best practices and optimizations.
    • Document ETL processes, data lineage, and metadata to facilitate knowledge sharing and compliance with data governance standards.

  • Big Data Solutions Expert

    1 day ago

    Kota, Rajasthan, India beBeeData Full time ₹ 8,00,000 - ₹ 15,00,000

    Unlock the Power of Data InsightsAbout the RoleThis exciting opportunity as a Big Data Developer will challenge you to create and manage robust data pipelines that handle large volumes of financial data.You will be responsible for designing, developing, and managing end-to-end data pipelines for various financial datasets.Experience with integrating...

  • Big Data Solutions Architect

    4 days ago

    Kota, Rajasthan, India beBeeDataEngineer Full time ₹ 2,50,00,000 - ₹ 3,00,00,000

    Job TitleA Data Engineer position is available to design, develop, and maintain data pipelines and ETL processes on Databricks.Maintain a high-performing and scalable data engineering environment.Collaborate with cross-functional teams to deliver actionable insights.Key RequirementsThe ideal candidate will have 7+ years of experience in Data Engineering and...

  • Big Data Engineer

    4 days ago

    Kota, Rajasthan, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Summary:We are seeking a seasoned Data Engineering Lead to helm a team of data engineers in designing and delivering scalable data pipelines and analytics platforms leveraging Microsoft Azure and Databricks.Key Responsibilities:Lead and grow a high-performing team of data engineers working on large-scale distributed data systems.Architect end-to-end data...

  • Big Data Analytics Project Coordinator

    22 hours ago

    Kota, Rajasthan, India beBeeProjectManager Full time ₹ 15,00,000 - ₹ 25,00,000

    The position requires a professional with experience in managing complex projects involving big data and analytics.Key responsibilities include:Delivering projects within the agreed-upon scope, timeline, and quality standardsEnsuring client satisfaction through effective communication and project managementManaging cross-functional teams to achieve project...

  • Senior Data Processing Specialist

    23 hours ago

    Kota, Rajasthan, India beBeeData Full time ₹ 12,00,000 - ₹ 25,00,000

    Big Data Engineer Job OpeningWe are seeking a skilled Big Data Engineer to join our team. As a Big Data Engineer, you will play a key role in designing, developing, and maintaining large-scale data processing systems.Key Responsibilities:Develop scalable data pipelines using PySpark, Hive, and other big data technologies.Evaluate data quality through the use...

  • Senior Data Platform Architect

    6 days ago

    Kota, Rajasthan, India beBeeSoftwareEngineer Full time US$ 18,00,000 - US$ 30,00,000

    We are seeking an exceptional software architect to join our team.The ideal candidate will have 8+ years of experience in software engineering, with a proven track record of architecting big data systems around technologies like Apache Spark and Hive.They will be responsible for developing a trusted, scalable and compliant platform to operate with speed,...

  • Senior Data Architect

    2 days ago

    Kota, Rajasthan, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Big Data Engineer Position OverviewWe are seeking a highly skilled Big Data Engineer to join our team. This individual will be responsible for designing, developing, and optimizing large-scale data pipelines and distributed data processing systems.Key Responsibilities:Design and develop scalable cloud-based data architectures.Work extensively with Scala,...

  • Chief Data Architect

    2 days ago

    Kota, Rajasthan, India beBeeData Full time ₹ 90,00,000 - ₹ 1,20,00,000

    Big Data SpecialistAs a Big Data specialist, you will be responsible for working with large datasets to uncover valuable insights and drive business growth.Key Responsibilities:Design and develop scalable data pipelines using big data technologies.Collaborate with cross-functional teams to implement data-driven solutions.Develop and maintain high-quality...

  • Mastering Data Engineering

    2 days ago

    Kota, Rajasthan, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Data Engineer Role SummaryWe are currently seeking a seasoned Senior Data Engineer with hands-on expertise in coding to help develop and implement an application for a major retail enterprise.The ideal candidate will possess extensive experience working with Big Data, Telemetry, and backend engineering as well as strong proficiency in SQL. Knowledge of IoT...

  • Lead Data Solutions Architect

    1 day ago

    Kota, Rajasthan, India beBeeData Full time US$ 1,00,000 - US$ 1,50,000

    Senior Data Pipeline SpecialistWe are seeking a seasoned Senior Data Pipeline Specialist to join our team. In this role, you will design, develop, and maintain scalable data pipelines and infrastructure that support real-time and batch data processing.About the Role:You will collaborate with cross-functional teams, including data scientists, analysts, and...