Databricks Spark Developer

2 weeks ago

coimbatore, India CGVAK Software AND Exports Limited Full time
Overview: TheDatabricks Spark Developer plays a crucial role in harnessing thepower of data using Databricks and Spark to develop and maintainefficient data pipelines. They are responsible for implementingscalable and reliable solutions that enable datadrivendecisionmaking within the organization.KeyResponsibilities:
  • Designing andimplementing robust ETL processes using Databricks andSpark
  • Developing and optimizing data pipelinesfor largescale data processing
  • Collaboratingwith data engineers and data scientists to support their datainfrastructure needs
  • Building and maintainingdata warehouse solutions to support businessanalytics
  • Performing data modeling andoptimization to ensure efficient data storage andretrieval
  • Troubleshooting and resolvingperformance issues with data infrastructure andpipelines
  • Implementing security and datagovernance best practices within the dataplatform
  • Automating data quality checks andensuring data consistency andaccuracy
  • Collaborating with crossfunctionalteams to understand data requirements and deliversolutions
  • Monitoring and maintaining the healthof data pipelines andinfrastructure
  • Documenting technical design andarchitecture of data solutions
  • Participating incode reviews and providing constructive feedback topeers
  • Staying updated with the latestadvancements in Databricks and Sparktechnologies
  • Providing technical guidance andmentorship to junior teammembers
RequiredQualifications:
  • Bachelors degree inComputer Science Engineering or a relatedfield
  • Proven experience in developing datapipelines using Databricks andSpark
  • Proficiency in ETL processes and datawarehousing concepts
  • Strong SQL skills with theability to write complex queries for data manipulation andanalysis
  • Advanced programming skills in Pythonfor data processing and manipulation
  • Experiencein data modeling and optimizing data storage forperformance
  • Deep understanding of big datatechnologies and distributed computingconcepts
  • Ability to troubleshoot and optimizedata pipeline performance for efficiency andreliability
  • Knowledge of data governancesecurity and compliance bestpractices
  • Excellent communication andcollaboration skills to work effectively in a teamenvironment
  • Strong analytical andproblemsolving abilities to tackle complex data engineeringchallenges
  • Ability to multitask and prioritizetasks in a fastpaced and dynamic workenvironment
  • Experience with cloud platformssuch as AWS Azure or GCP is aplus
  • Certifications in Databricks andSparkrelated technologies aredesirable
  • Experience in Agile developmentmethodologies and version controlsystems

ExperienceRequired:Data Engineeringresource with Databricks (Databricks develops a webbased platformfor working with Spark) experience with Python coding background& SQL

spark,python,datainfrastructure,data modeling

  • Databricks Spark Developer

    2 weeks ago

    Coimbatore, India CGVAK Software AND Exports Limited Full time

    Overview: TheDatabricks Spark Developer plays a crucial role in harnessing thepower of data using Databricks and Spark to develop and maintainefficient data pipelines. They are responsible for implementingscalable and reliable solutions that enable datadrivendecisionmaking within the organization. KeyResponsibilities:Designing andimplementing robust ETL...

  • Databricks Spark Developer

    4 weeks ago

    Coimbatore, India CGVAK Software AND Exports Limited Full time

    Overview:TheDatabricks Spark Developer plays a crucial role in harnessing thepower of data using Databricks and Spark to develop and maintainefficient data pipelines. They are responsible for implementingscalable and reliable solutions that enable datadrivendecisionmaking within the organization.KeyResponsibilities:Designing andimplementing robust ETL...

  • Data Engineer

    4 weeks ago

    Coimbatore, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    Job DescriptionPrimary Skills : Databricks with Pyspark, PythonSecondary skills; ADF- 6+ years of experience with detailed knowledge of data warehouse technical architectures, ETL/ ELT and reporting/analytic tools.- 4+ years of work experience with very large data warehousing environment- 4+ years of work experience in Databricks, Pyspark and Python project...

  • Data Engineer

    3 weeks ago

    Coimbatore, Tamil Nadu, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    Job DescriptionPrimary Skills : Databricks with Pyspark, PythonSecondary skills; ADF- 6+ years of experience with detailed knowledge of data warehouse technical architectures, ETL/ ELT and reporting/analytic tools.- 4+ years of work experience with very large data warehousing environment- 4+ years of work experience in Databricks, Pyspark and Python project...

  • Data Engineer

    1 month ago

    Coimbatore, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    Job DescriptionPrimary Skills : Databricks with Pyspark, PythonSecondary skills; ADF- 6+ years of experience with detailed knowledge of data warehouse technical architectures, ETL/ ELT and reporting/analytic tools.- 4+ years of work experience with very large data warehousing environment- 4+ years of work experience in Databricks, Pyspark and Python project...

  • Data Engineer

    5 days ago

    coimbatore, India CGVAK Software AND Exports Limited Full time

    Overview: TheDatabricks Spark Developer plays a crucial role in harnessing thepower of data using Databricks and Spark to develop and maintainefficient data pipelines. They are responsible for implementingscalable and reliable solutions that enable datadrivendecisionmaking within the organization.KeyResponsibilities: Designing andimplementing robust ETL...

  • Data Engineer

    4 weeks ago

    Coimbatore, India CGVAK Software AND Exports Limited Full time

    Overview:TheDatabricks Spark Developer plays a crucial role in harnessing thepower of data using Databricks and Spark to develop and maintainefficient data pipelines. They are responsible for implementingscalable and reliable solutions that enable datadrivendecisionmaking within the organization.KeyResponsibilities:Designing andimplementing robust ETL...

  • Data Engineer

    1 month ago

    Coimbatore, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    Job Description : - Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.- Diagnose system...

  • Data Engineer

    4 weeks ago

    Coimbatore, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    Job Description :- Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.- Diagnose system performance...

  • Data Engineer

    3 weeks ago

    Coimbatore, Tamil Nadu, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    Job Description :- Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.- Diagnose system performance...

  • Senior Data Engineer

    2 weeks ago

    Coimbatore, India PureSearch Full time

    Job Description : We are seeking a talented Senior Data Engineer with 3 to 5 years of experience to join the client's team. The ideal candidate will have strong expertise in SQL and ETL processes using tools such as Pentaho, Talend, Azure Data Factory (ADF), Databricks, or AWS Glue. Familiarity with event-driven architecture patterns such as Lambda,...

  • Senior Data Engineer

    2 weeks ago

    Coimbatore, India PureSearch Full time

    Job Description : We are seeking a talented Senior Data Engineer with 3 to 5 years of experience to join the client's team. The ideal candidate will have strong expertise in SQL and ETL processes using tools such as Pentaho, Talend, Azure Data Factory (ADF), Databricks, or AWS Glue. Familiarity with event-driven architecture patterns such as Lambda,...

  • Senior Data Engineer

    1 month ago

    coimbatore, India PureSearch Full time

    Job Description : We are seeking a talented Senior Data Engineer with 3 to 5 years of experience to join the client's team. The ideal candidate will have strong expertise in SQL and ETL processes using tools such as Pentaho, Talend, Azure Data Factory (ADF), Databricks, or AWS Glue. Familiarity with event-driven architecture patterns such as Lambda,...

  • Senior Data Engineer

    3 weeks ago

    Coimbatore, Tamil Nadu, India PureSearch Full time

    Job Description : We are seeking a talented Senior Data Engineer with 3 to 5 years of experience to join the client's team. The ideal candidate will have strong expertise in SQL and ETL processes using tools such as Pentaho, Talend, Azure Data Factory (ADF), Databricks, or AWS Glue. Familiarity with event-driven architecture patterns such as Lambda,...

  • Data Engineering Lead

    2 weeks ago

    Coimbatore, India BlueByte Technologies Full time

    Job Title : Data Engineering Lead (With Architecture Experience)Location : CoimbatorePosition Type : Full-Time Key Responsibilities :Architect and Design Data Solutions :- Collaborate with cross-functional teams to understand business requirements and design scalable and efficient data solutions.- Define data architectures, data models & data integration...

  • Data Engineering Lead

    1 month ago

    coimbatore, India BlueByte Technologies Full time

    Job Title : Data Engineering Lead (With Architecture Experience)Location : CoimbatorePosition Type : Full-Time Key Responsibilities :Architect and Design Data Solutions :- Collaborate with cross-functional teams to understand business requirements and design scalable and efficient data solutions.- Define data architectures, data models & data integration...

  • Data Engineering Lead

    2 weeks ago

    Coimbatore, India BlueByte Technologies Full time

    Job Title : Data Engineering Lead (With Architecture Experience)Location : CoimbatorePosition Type : Full-Time Key Responsibilities :Architect and Design Data Solutions :- Collaborate with cross-functional teams to understand business requirements and design scalable and efficient data solutions.- Define data architectures, data models & data integration...

  • Data Engineering Lead

    3 weeks ago

    Coimbatore, Tamil Nadu, India BlueByte Technologies Full time

    Job Title : Data Engineering Lead (With Architecture Experience)Location : CoimbatorePosition Type : Full-Time Key Responsibilities :Architect and Design Data Solutions :- Collaborate with cross-functional teams to understand business requirements and design scalable and efficient data solutions.- Define data architectures, data models & data integration...

  • Spark/PySpark Developer

    1 month ago

    Bihar,Jharkhand,Maharashtra,Pondicherry,Coimbatore,Patna,Aurangabad,Ranchi,Mumbai,Navi Mumbai,Pune,N, India ATech Full time

    Job Profile : Spark ( Pyspark ) DeveloperIndustry Type : IT Services Job description :- The developer must have sound knowledge in Apache Spark and Python programming.- Deep experience in developing data processing tasks using pySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations.-...

  • Spark/PySpark Developer

    3 weeks ago

    Bihar/Jharkhand/Maharashtra/Pondicherry/Coimbatore/Patna/Aurangabad/Ranchi/Mumbai/Navi Mumbai/Pune/N, IN ATech Full time

    Job Profile : Spark ( Pyspark ) DeveloperIndustry Type : IT Services Job description :- The developer must have sound knowledge in Apache Spark and Python programming.- Deep experience in developing data processing tasks using pySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations.-...