Current jobs related to Advanced Big Data Architect - Nashik, Maharashtra - beBeeData

  • Data Strategy Architect

    1 week ago

    Nashik, Maharashtra, India beBeeData Full time ₹ 30,00,000 - ₹ 40,00,000

    Job Title: Data Strategy ArchitectWe are seeking a seasoned Data Strategy Architect to lead the design and implementation of advanced analytics solutions for our clients. The ideal candidate will have a strong background in data architecture, cloud computing, and big data technologies.About the Role:

  • Architect - Big data and Cloud

    2 weeks ago

    Nashik, Maharashtra, India Societe Generale Global Solution Centre Full time

    Role: Cloud and Big data ArchitectRole DescriptionWe are looking for skilled and passionate Azure Cloud experts with a strong focus on Big Data management and data quality skills to join our team at Société Générale Global Solution Centre in Bengaluru.This is a full-time hybrid role for Digital & Data Solutions (DDS), offering flexibility for remote...

  • Big Data Architect

    1 week ago

    Nashik, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,25,00,000

    Job Title: Big Data Architect - Cloud and Scalability ExpertOur team is seeking a highly skilled Big Data Architect to lead the design, implementation, and maintenance of large-scale data systems. Key Responsibilities: Design and implement robust data pipelines using Python, SQL, PySpark, and bash scripts to extract insights from complex datasets....

  • Senior Big Data Engineer

    2 weeks ago

    Nashik, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 17,00,000

    Job Title: Advanced Data Solutions ArchitectJob Summary:We seek an experienced data architect to help build and optimize high-performance data solutions in a scalable environment. The ideal candidate has hands-on experience with Databricks production environments and a deep understanding of modern data architecture.Key Responsibilities:Design, develop, and...

  • Data Architect

    2 weeks ago

    Nashik, Maharashtra, India beBeeData Full time ₹ 1,00,00,000 - ₹ 2,50,00,000

    Job Title: Data ArchitectOverview:We are seeking an experienced Data Architect to lead the development of our big data ecosystem. As a key member of our team, you will be responsible for designing and implementing scalable data solutions that drive business growth.Key Responsibilities:Design and develop distributed computing systems using Apache Spark and...

  • Big Data Engineering Lead

    2 weeks ago

    Nashik, Maharashtra, India beBeeDataEngineering Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Job Overview:We are seeking an experienced Big Data Engineering Lead to lead our data engineering teams and drive the development of scalable data infrastructure. The successful candidate will be responsible for managing cross-functional collaboration, developing data pipelines, and provisioning data infrastructure.About You:You have 7-12 years of experience...

  • Big Data Developer

    2 weeks ago

    Nashik, Maharashtra, India beBeeDataScience Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data ScientistLocation: Nashik, IndiaAbout the Role:Main ResponsibilitiesDevelop and implement big data solutions using Scala, Spark, and related technologies.Create and optimize data pipelines, transformation logic, and data models for large-scale data processing.Work on implementation projects with a focus on big data analytics and...

  • Big Data Specialist

    1 week ago

    Nashik, Maharashtra, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    About the Role:We are seeking a highly skilled Big Data Administrator to manage our big data platform.The ideal candidate will have experience with Hadoop, Hive, HBase, BigSQL, and other big data technologies. They will be responsible for administering and monitoring big data platform components, handling day-to-day operations, and supporting technical and...

  • Senior Data Architect

    2 weeks ago

    Nashik, Maharashtra, India beBeeData Full time ₹ 90,00,000 - ₹ 1,21,00,000

    Job Title: Senior Data ArchitectAs a Senior Data Architect, you will play a key role in designing and building scalable data pipelines that support business growth. The ideal candidate will have expertise in cloud technologies, particularly AWS, and experience with big data processing tools such as Glue, EMR/Spark, and Kafka/Kinesis streaming.The successful...

  • Senior Data Architect Position

    2 weeks ago

    Nashik, Maharashtra, India beBeeDataArchitect Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job DescriptionWe are seeking a seasoned Data Architect to design and develop features in our existing Data Warehouse.This role is responsible for establishing connections between Engineering, product, and analytics/data scientists teams.The ideal candidate will implement and update batch ETL pipelines, define data architecture, and partner with engineers...

Advanced Big Data Architect

3 weeks ago

Nashik, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

Data Engineer Job Description

We are seeking a highly skilled Data Architect to join our data platform implementation team. In this role, you will be instrumental in designing, developing, and maintaining ETL processes to ensure efficient extraction, transformation, and loading of data from various sources into our data lake and data warehouse.

Key Responsibilities:

  • Design and develop ETL processes using Databricks PySpark to extract, transform, and load data from heterogeneous sources into our data lake and data warehouse.
  • Optimize ETL workflows for performance and scalability, leveraging Databricks PySpark and Spark SQL to efficiently process large data volumes.
  • Implement robust error handling and monitoring mechanisms to proactively detect and resolve issues within ETL processes.
  • Design and implement data solutions following the Medallion Architecture principles, organizing data into Bronze, Silver, and Gold layers.
  • Ensure data is appropriately cleansed, enriched, and optimized at each stage to support robust analytics and reporting.

Data Pipeline Management:

  • Hands On experience in creating advanced data pipelines using databricks workflows
  • Develop and maintain data pipelines using Databricks PySpark, ensuring data quality, integrity, and reliability throughout the ETL lifecycle.
  • Collaborate with data engineering, data science, and business intelligence teams to translate data requirements into efficient ETL workflows and pipelines.

Data Analysis and Query Optimization:

  • Write and optimize complex SQL queries for data manipulation, aggregation, and analysis within Databricks PySpark applications.

Project Coordination and Continuous Improvement:

  • Participate in project planning and coordination activities to ensure timely delivery of ETL solutions.
  • Stay updated on the latest developments in Databricks PySpark, Spark SQL, and related technologies, recommending and implementing best practices and optimizations.
  • Document ETL processes, data lineage, and metadata to facilitate knowledge sharing and ensure compliance with data governance standards.

Requirements:

  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • Minimum of 3 years of experience as an ETL developer, with a strong focus on Databricks PySpark development.
  • Proficiency in Python programming, with extensive experience in developing and debugging Databricks PySpark applications.
  • In-depth understanding of Spark architecture and internals, with hands-on experience in Spark RDDs, DataFrames, and Spark SQL.
  • Expertise in writing and optimizing complex SQL queries for data manipulation, aggregation, and analysis.
  • Proven experience in working with large-scale data warehousing and ETL frameworks.
  • Strong problem-solving skills and the ability to troubleshoot and resolve ETL process issues.
  • Excellent communication and collaboration skills, with the ability to work effectively in a team environment.

Preferred Qualifications:

  • Experience with cloud platforms with a preference for AWS.
  • Experience with data platform tools such as DataBricks, Snowflake, and Tableau.
  • Demonstrated ability to implement best practices for ETL processes and data management.
  • Strong understanding of data governance and data quality principles.
  • Relevant certifications in Databricks PySpark, Spark SQL, or related technologies.