Data Solutions Architect

3 days ago

Coimbatore, Tamil Nadu, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 15,00,000
Data Engineer Position

We are seeking a skilled Data Engineer to join our organization. The successful candidate will be responsible for designing, developing, and maintaining efficient ETL processes using Databricks PySpark.

Key Responsibilities:

  • ETL Development:
    • Design and develop ETL processes using Databricks PySpark to extract, transform, and load data from heterogeneous sources into our data lake and data warehouse.
    • Optimize ETL workflows for performance and scalability, leveraging Databricks PySpark and Spark SQL to efficiently process large data volumes.
    • Implement robust error handling and monitoring mechanisms to proactively detect and resolve issues within ETL processes.
    • Design and implement data solutions following the Medallion Architecture principles, organizing data into Bronze, Silver, and Gold layers.
    • Ensure data is appropriately cleansed, enriched, and optimized at each stage to support robust analytics and reporting.
  • Data Pipeline Management:
    • Create advanced data pipelines using databricks workflows
    • Develop and maintain data pipelines using Databricks PySpark, ensuring data quality, integrity, and reliability throughout the ETL lifecycle.
    • Collaborate with data engineering, data science, and business intelligence teams to translate data requirements into efficient ETL workflows and pipelines.
  • Data Analysis and Query Optimization:
    • Write and optimize complex SQL queries for data manipulation, aggregation, and analysis within Databricks PySpark applications.
  • Project Coordination and Continuous Improvement:
    • Participate in project planning and coordination activities to ensure timely delivery of ETL solutions.
    • Stay updated on the latest developments in Databricks PySpark, Spark SQL, and related technologies, recommending and implementing best practices and optimizations.
    • Document ETL processes, data lineage, and metadata to facilitate knowledge sharing and ensure compliance with data governance standards.

Required Skills and Qualifications:

  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • Minimum of 3 years of experience as an ETL developer, with a strong focus on Databricks PySpark development.
  • Proficiency in Python programming, with extensive experience in developing and debugging Databricks PySpark applications.
  • In-depth understanding of Spark architecture and internals, with hands-on experience in Spark RDDs, DataFrames, and Spark SQL.
  • Expertise in writing and optimizing complex SQL queries for data manipulation, aggregation, and analysis.
  • Proven experience in working with large-scale data warehousing and ETL frameworks.
  • Strong problem-solving skills and the ability to troubleshoot and resolve ETL process issues.
  • Excellent communication and collaboration skills, with the ability to work effectively in a team environment.

Preferred Qualifications:

  • Experience with cloud platforms with a preference for AWS.
  • Experience with data platform tools such as DataBricks, Snowflake, and Tableau.
  • Demonstrated ability to implement best practices for ETL processes and data management.
  • Strong understanding of data governance and data quality principles.
  • Relevant certifications in Databricks PySpark, Spark SQL, or related technologies.

  • Cloud Data Solutions Architect

    4 days ago

    Coimbatore, Tamil Nadu, India beBeeData Full time ₹ 18,00,000 - ₹ 20,00,000

    Job Title: Cloud Data Solutions Architect We are seeking a highly skilled and experienced Cloud Data Solutions Architect to lead the design and implementation of scalable and secure data solutions. The ideal candidate will have a deep understanding of Microsoft Azure services, including Power BI, Synapse, and Data Factory.Responsibilities:Design and oversee...

  • Snowflake Data Solutions Architect

    4 days ago

    Coimbatore, Tamil Nadu, India beBeeDataScientist Full time ₹ 15,03,600 - ₹ 21,45,200

    About the RoleSeeking an IT Professional with expertise in Snowflake Data Cloud to lead design, implementation, and optimization of scalable data solutions.Demonstrate deep understanding of data warehousing concepts and work with diverse data structures.Key Responsibilities:Architect end-to-end data solutions on Snowflake.Develop and manage API-based...

  • Data Architect

    6 days ago

    Coimbatore, Tamil Nadu, India Hash Agile Technologies Full time

    About UsAt Hash Agile Technologies, we empower businesses with data-driven insights by building scalable, secure, and innovative data solutions. We are looking for a highly skilled Data Architect to design and lead our data ecosystem, ensuring seamless integration, accessibility, and reliability across platforms.Key Responsibilities- Design and implement...

  • Data Architect

    4 days ago

    Coimbatore, Tamil Nadu, India Hash Agile Technologies Full time

    About Us At Hash Agile Technologies, we empower businesses with data-driven insights by building scalable, secure, and innovative data solutions. We are looking for a highly skilled Data Architect to design and lead our data ecosystem, ensuring seamless integration, accessibility, and reliability across platforms. Key Responsibilities Design and implement...

  • Data Architect

    1 week ago

    Coimbatore, Tamil Nadu, India Hash Agile Technologies Full time

    About UsAt Hash Agile Technologies, we empower businesses with data-driven insights by building scalable, secure, and innovative data solutions. We are looking for a highly skilled Data Architect to design and lead our data ecosystem, ensuring seamless integration, accessibility, and reliability across platforms.Key ResponsibilitiesDesign and implement ...

  • Chief Data Solutions Architect

    2 days ago

    Coimbatore, Tamil Nadu, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Enterprise Data EngineerWe are seeking a skilled Enterprise Data Engineer to design and implement scalable data solutions using PostgreSQL and cloud-based technologies.The ideal candidate has expertise in SQL engineering, cloud data architectures, and modern data pipelines. Experience with hosted data warehouse platforms is beneficial.Design and build a...

  • AI Solutions Architect

    4 days ago

    Coimbatore, Tamil Nadu, India beBeeDataEngineer Full time US$ 1,50,000 - US$ 2,50,000

    **Job Title:**AI Solutions ArchitectWe are seeking a highly skilled AI Solutions Architect with expertise in designing and building scalable data pipelines. As a critical member of our AI team, you will be responsible for creating innovative solutions to integrate healthcare data.Design, build, and operate the backbone of our AI-powered...

  • Senior Data Solutions Designer

    3 days ago

    Coimbatore, Tamil Nadu, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Description", "We are seeking a highly skilled Data Architect to design and implement scalable, secure, and efficient data solutions." ,"Develop data pipelines, architect data lakes, and implement data warehousing solutions using Databricks.Collaborate with data scientists and analysts to develop and deploy machine learning models and analytics solutions...

  • Chief Data Solutions Architect

    2 days ago

    Coimbatore, Tamil Nadu, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    About Aezion.A trusted digital engineering provider in the USA. We live by the adage that our word is our bond.We achieve this by investing effort to exceed client expectations throughout the project lifecycle: Design, Delivery, Validation, and Stewardship.We believe work is ministry, an expression of our values. Our goal is to honor commitments to clients...

  • Data Solutions Expert

    6 days ago

    Coimbatore, Tamil Nadu, India beBeeDataDriven Full time ₹ 20,00,000 - ₹ 25,00,000

    About UsWe provide data-driven solutions to empower businesses. Our team of experts designs and implements scalable, secure, and innovative data architecture.Key ResponsibilitiesWe design end-to-end data architecture solutions that meet business needs.Our team architects, builds, and optimizes data pipelines and ETL workflows using industry-leading tools.We...