Databricks Spark Developer

2 weeks ago

Coimbatore, India CGVAK Software AND Exports Limited Full time
Overview: TheDatabricks Spark Developer plays a crucial role in harnessing thepower of data using Databricks and Spark to develop and maintainefficient data pipelines. They are responsible for implementingscalable and reliable solutions that enable datadrivendecisionmaking within the organization. KeyResponsibilities:

Designing andimplementing robust ETL processes using Databricks andSparkDeveloping and optimizing data pipelinesfor largescale data processingCollaboratingwith data engineers and data scientists to support their datainfrastructure needsBuilding and maintainingdata warehouse solutions to support businessanalyticsPerforming data modeling andoptimization to ensure efficient data storage andretrievalTroubleshooting and resolvingperformance issues with data infrastructure andpipelinesImplementing security and datagovernance best practices within the dataplatformAutomating data quality checks andensuring data consistency andaccuracyCollaborating with crossfunctionalteams to understand data requirements and deliversolutionsMonitoring and maintaining the healthof data pipelines andinfrastructureDocumenting technical design andarchitecture of data solutionsParticipating incode reviews and providing constructive feedback topeersStaying updated with the latestadvancements in Databricks and SparktechnologiesProviding technical guidance andmentorship to junior teammembersRequiredQualifications:

Bachelors degree inComputer Science Engineering or a relatedfieldProven experience in developing datapipelines using Databricks andSparkProficiency in ETL processes and datawarehousing conceptsStrong SQL skills with theability to write complex queries for data manipulation andanalysisAdvanced programming skills in Pythonfor data processing and manipulationExperiencein data modeling and optimizing data storage forperformanceDeep understanding of big datatechnologies and distributed computingconceptsAbility to troubleshoot and optimizedata pipeline performance for efficiency andreliabilityKnowledge of data governancesecurity and compliance bestpracticesExcellent communication andcollaboration skills to work effectively in a teamenvironmentStrong analytical andproblemsolving abilities to tackle complex data engineeringchallengesAbility to multitask and prioritizetasks in a fastpaced and dynamic workenvironmentExperience with cloud platformssuch as AWS Azure or GCP is aplusCertifications in Databricks andSparkrelated technologies aredesirableExperience in Agile developmentmethodologies and version controlsystems

ExperienceRequired: Data Engineeringresource with Databricks (Databricks develops a webbased platformfor working with Spark) experience with Python coding background& SQLspark,python,datainfrastructure,data modeling

  • Databricks Spark Developer

    4 weeks ago

    Coimbatore, India CGVAK Software AND Exports Limited Full time

    Overview:TheDatabricks Spark Developer plays a crucial role in harnessing thepower of data using Databricks and Spark to develop and maintainefficient data pipelines. They are responsible for implementingscalable and reliable solutions that enable datadrivendecisionmaking within the organization.KeyResponsibilities:Designing andimplementing robust ETL...

  • Databricks Spark Developer

    2 weeks ago

    coimbatore, India CGVAK Software AND Exports Limited Full time

    Overview: TheDatabricks Spark Developer plays a crucial role in harnessing thepower of data using Databricks and Spark to develop and maintainefficient data pipelines. They are responsible for implementingscalable and reliable solutions that enable datadrivendecisionmaking within the organization.KeyResponsibilities: Designing andimplementing robust ETL...

  • Data Engineer

    4 weeks ago

    Coimbatore, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    Job DescriptionPrimary Skills : Databricks with Pyspark, PythonSecondary skills; ADF- 6+ years of experience with detailed knowledge of data warehouse technical architectures, ETL/ ELT and reporting/analytic tools.- 4+ years of work experience with very large data warehousing environment- 4+ years of work experience in Databricks, Pyspark and Python project...

  • Data Engineer

    3 weeks ago

    Coimbatore, Tamil Nadu, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    Job DescriptionPrimary Skills : Databricks with Pyspark, PythonSecondary skills; ADF- 6+ years of experience with detailed knowledge of data warehouse technical architectures, ETL/ ELT and reporting/analytic tools.- 4+ years of work experience with very large data warehousing environment- 4+ years of work experience in Databricks, Pyspark and Python project...

  • Data Engineer

    1 month ago

    Coimbatore, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    Job DescriptionPrimary Skills : Databricks with Pyspark, PythonSecondary skills; ADF- 6+ years of experience with detailed knowledge of data warehouse technical architectures, ETL/ ELT and reporting/analytic tools.- 4+ years of work experience with very large data warehousing environment- 4+ years of work experience in Databricks, Pyspark and Python project...

  • Data Engineer

    5 days ago

    coimbatore, India CGVAK Software AND Exports Limited Full time

    Overview: TheDatabricks Spark Developer plays a crucial role in harnessing thepower of data using Databricks and Spark to develop and maintainefficient data pipelines. They are responsible for implementingscalable and reliable solutions that enable datadrivendecisionmaking within the organization.KeyResponsibilities: Designing andimplementing robust ETL...

  • Data Engineer

    4 weeks ago

    Coimbatore, India CGVAK Software AND Exports Limited Full time

    Overview:TheDatabricks Spark Developer plays a crucial role in harnessing thepower of data using Databricks and Spark to develop and maintainefficient data pipelines. They are responsible for implementingscalable and reliable solutions that enable datadrivendecisionmaking within the organization.KeyResponsibilities:Designing andimplementing robust ETL...

  • Data Engineer

    4 weeks ago

    Coimbatore, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    Job Description :- Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.- Diagnose system performance...

  • Data Engineer

    1 month ago

    Coimbatore, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    Job Description : - Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.- Diagnose system...

  • Data Engineer

    3 weeks ago

    Coimbatore, Tamil Nadu, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    Job Description :- Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.- Diagnose system performance...

  • Senior Data Engineer

    2 weeks ago

    Coimbatore, India PureSearch Full time

    Job Description : We are seeking a talented Senior Data Engineer with 3 to 5 years of experience to join the client's team. The ideal candidate will have strong expertise in SQL and ETL processes using tools such as Pentaho, Talend, Azure Data Factory (ADF), Databricks, or AWS Glue. Familiarity with event-driven architecture patterns such as Lambda,...

  • Senior Data Engineer

    2 weeks ago

    Coimbatore, India PureSearch Full time

    Job Description : We are seeking a talented Senior Data Engineer with 3 to 5 years of experience to join the client's team. The ideal candidate will have strong expertise in SQL and ETL processes using tools such as Pentaho, Talend, Azure Data Factory (ADF), Databricks, or AWS Glue. Familiarity with event-driven architecture patterns such as Lambda,...

  • Senior Data Engineer

    1 month ago

    coimbatore, India PureSearch Full time

    Job Description : We are seeking a talented Senior Data Engineer with 3 to 5 years of experience to join the client's team. The ideal candidate will have strong expertise in SQL and ETL processes using tools such as Pentaho, Talend, Azure Data Factory (ADF), Databricks, or AWS Glue. Familiarity with event-driven architecture patterns such as Lambda,...

  • Senior Data Engineer

    3 weeks ago

    Coimbatore, Tamil Nadu, India PureSearch Full time

    Job Description : We are seeking a talented Senior Data Engineer with 3 to 5 years of experience to join the client's team. The ideal candidate will have strong expertise in SQL and ETL processes using tools such as Pentaho, Talend, Azure Data Factory (ADF), Databricks, or AWS Glue. Familiarity with event-driven architecture patterns such as Lambda,...

  • Data Engineering Lead

    2 weeks ago

    Coimbatore, India BlueByte Technologies Full time

    Job Title : Data Engineering Lead (With Architecture Experience)Location : CoimbatorePosition Type : Full-Time Key Responsibilities :Architect and Design Data Solutions :- Collaborate with cross-functional teams to understand business requirements and design scalable and efficient data solutions.- Define data architectures, data models & data integration...

  • Data Engineering Lead

    2 weeks ago

    Coimbatore, India BlueByte Technologies Full time

    Job Title : Data Engineering Lead (With Architecture Experience)Location : CoimbatorePosition Type : Full-Time Key Responsibilities :Architect and Design Data Solutions :- Collaborate with cross-functional teams to understand business requirements and design scalable and efficient data solutions.- Define data architectures, data models & data integration...

  • Data Engineering Lead

    1 month ago

    coimbatore, India BlueByte Technologies Full time

    Job Title : Data Engineering Lead (With Architecture Experience)Location : CoimbatorePosition Type : Full-Time Key Responsibilities :Architect and Design Data Solutions :- Collaborate with cross-functional teams to understand business requirements and design scalable and efficient data solutions.- Define data architectures, data models & data integration...

  • Data Engineering Lead

    3 weeks ago

    Coimbatore, Tamil Nadu, India BlueByte Technologies Full time

    Job Title : Data Engineering Lead (With Architecture Experience)Location : CoimbatorePosition Type : Full-Time Key Responsibilities :Architect and Design Data Solutions :- Collaborate with cross-functional teams to understand business requirements and design scalable and efficient data solutions.- Define data architectures, data models & data integration...

  • Spark/PySpark Developer

    1 month ago

    Bihar,Jharkhand,Maharashtra,Pondicherry,Coimbatore,Patna,Aurangabad,Ranchi,Mumbai,Navi Mumbai,Pune,N, India ATech Full time

    Job Profile : Spark ( Pyspark ) DeveloperIndustry Type : IT Services Job description :- The developer must have sound knowledge in Apache Spark and Python programming.- Deep experience in developing data processing tasks using pySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations.-...

  • Spark/PySpark Developer

    3 weeks ago

    Bihar/Jharkhand/Maharashtra/Pondicherry/Coimbatore/Patna/Aurangabad/Ranchi/Mumbai/Navi Mumbai/Pune/N, IN ATech Full time

    Job Profile : Spark ( Pyspark ) DeveloperIndustry Type : IT Services Job description :- The developer must have sound knowledge in Apache Spark and Python programming.- Deep experience in developing data processing tasks using pySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations.-...