Senior Data Solutions Developer

19 hours ago

Anand, Gujarat, India beBeeEngineer Full time ₹ 15,00,000 - ₹ 25,00,000
Big Data Engineer

We are seeking a highly skilled data engineer to join our big data platform team. As a key member of the team, you will be responsible for designing and developing scalable ETL processes using Databricks PySpark to extract, transform, and load data from various sources into our data lake and data warehouse.

Key Responsibilities:

  1. ETL Process Development: Design and develop ETL processes using Databricks PySpark to efficiently extract, transform, and load data from heterogeneous sources into our data lake and data warehouse.
  2. ETL Workflow Optimization: Optimize ETL workflows for performance and scalability, leveraging Databricks PySpark and Spark SQL to efficiently process large data volumes.
  3. Robust Error Handling: Implement robust error handling and monitoring mechanisms to proactively detect and resolve issues within ETL processes.
  4. Data Solution Implementation: Design and implement data solutions following the Medallion Architecture principles, organizing data into Bronze, Silver, and Gold layers.
  5. Data Quality: Ensure data is appropriately cleansed, enriched, and optimized at each stage to support robust analytics and reporting.

Required Skills and Qualifications:

  • Education: Bachelor's degree in Computer Science, Engineering, or a related field.
  • Experience: Minimum of 3 years of experience as an ETL developer, with a strong focus on Databricks PySpark development.
  • Programming Skills: Proficiency in Python programming, with extensive experience in developing and debugging Databricks PySpark applications.
  • Spark Knowledge: In-depth understanding of Spark architecture and internals, with hands-on experience in Spark RDDs, DataFrames, and Spark SQL.
  • SQL Expertise: Expertise in writing and optimizing complex SQL queries for data manipulation, aggregation, and analysis.
  • Large-Scale Data Experience: Proven experience in working with large-scale data warehousing and ETL frameworks.
  • Problem-Solving Skills: Strong problem-solving skills and the ability to troubleshoot and resolve ETL process issues.
  • Communication Skills: Excellent communication and collaboration skills, with the ability to work effectively in a team environment.

Preferred Qualifications:

  • Cloud Platform Experience: Experience with cloud platforms with a preference for AWS.
  • Data Platform Tools: Experience with data platform tools such as DataBricks, Snowflake, and Tableau.
  • Best Practices: Demonstrated ability to implement best practices for ETL processes and data management.
  • Data Governance: Strong understanding of data governance and data quality principles.
  • Certifications: Relevant certifications in Databricks PySpark, Spark SQL, or related technologies.

 

  • Data Solutions Specialist

    2 days ago

    Anand, Gujarat, India beBeeDatabase Full time ₹ 12,00,000 - ₹ 15,00,000

    An innovative Data Solutions Specialist plays a pivotal role in driving business growth through cutting-edge database development.This strategic role requires extensive knowledge of various industry processes and will ensure seamless business operations by managing data infrastructure and coaching technical teams.Participate in daily data management to...

  • Senior Data Developer

    6 days ago

    Anand, Gujarat, India beBeeData Full time US$ 80,000 - US$ 1,10,000

    OverviewThis role focuses on supporting the development and implementation of data models, with a key emphasis on Machine Learning. The ideal candidate will be working under the supervision of more experienced scientists to contribute to innovative projects.Job Description:Data Model Development: Assist in the design and implementation of data-driven...

  • Senior Data-Driven Solutions Developer

    1 day ago

    Anand, Gujarat, India beBeeDataDriven Full time ₹ 15,00,000 - ₹ 25,00,000

    Job DescriptionWe are seeking a talented and experienced professional to join our team as a Marketing Technology Developer. In this role, you will be responsible for designing and building scalable solutions using Microsoft Dynamics 365 CE, PowerApps, and Power Automate.You will work closely with business and IT stakeholders to align data structures and...

  • Data Analytics Solutions Developer

    15 hours ago

    Anand, Gujarat, India beBeeBusinessIntelligenceDeveloper Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Business Intelligence Developer">">Key Responsibilities:Collaborate with stakeholders to gather and analyze business requirements.Work with cross-functional teams to design, develop, and implement data analytics solutions.Design and develop technical solutions that meet business needs.Ensure data quality, consistency, and performance in the new...

  • Advanced Business Solution Developer

    14 hours ago

    Anand, Gujarat, India beBeeClarity Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Clarity Developer OpportunityAbout the Role:We are seeking a seasoned Senior Clarity Developer to spearhead advanced business solution development and implementation using Duck Creek Clarity and other technologies. Key responsibilities include designing, developing, and implementing custom Clarity solutions, collaborating with various platforms and...

  • Senior Network and Data Solutions Manager

    19 hours ago

    Anand, Gujarat, India beBeeConnectivity Full time ₹ 1,00,00,000 - ₹ 1,50,00,000

    Job Title: Senior Network and Data Solutions ManagerJob DescriptionThis role leads the transformation of wireline and enterprise connectivity business into a future-fit, converged services portfolio by integrating connectivity with security and managed services.The job holder will own the full product P&L and drive growth through strategy, product...

  • Senior Data Solutions Architect

    19 hours ago

    Anand, Gujarat, India beBeeData Full time ₹ 16,56,000 - ₹ 25,12,000

    About the Role:We are seeking a skilled Data Engineer to design, build, and optimize scalable data solutions using Google Cloud Platform (GCP) and AtScale.The ideal candidate will have hands-on experience in developing robust data pipelines, implementing efficient data models, and ensuring performance optimization for enterprise-level analytics.

  • Enterprise Data Solutions Architect

    3 hours ago

    Anand, Gujarat, India beBeeCloud Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Enterprise Data Solutions ArchitectAs a Senior Data Engineer, you will lead the development of data architectures that meet client needs. You will provide technical leadership and guidance to your team.Key Responsibilities:• Develop pipelines to integrate diverse data sources, including social media and public datasets.• Collaborate with...

  • Senior Data Engineer

    19 hours ago

    Anand, Gujarat, India beBeeDataEngineer Full time ₹ 80,00,000 - ₹ 1,50,00,000

    Job Title: Senior Data EngineerWe are seeking a highly skilled and experienced Senior Data Engineer to join our team.Design, develop, and maintain large-scale data pipelines using PySparkCollaborate with cross-functional teams to integrate data into cloud-based data platformsDevelop and implement data processing solutions using Spark SQL, DataFrame, and RDD...

  • Highly Skilled Data Solutions Developer

    6 days ago

    Anand, Gujarat, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 15,00,000

    Job Title: Data EngineerKey Responsibilities:Design and develop large-scale data engineering solutions using Informatica DEI.Collaborate with cross-functional teams to integrate data from various sources.Required Skills:Expertise in data engineering integration using Informatica DEI.Proficiency in Azure Data Engineering stack, including Databricks, Synapse,...