Spark Scala Data Engineer

2 weeks ago

Chennai, Tamil Nadu, India CAPCO Full time
Spark Scala Data Engineer - Bangalore/ Chennai

at Capco India

Job Title:
Data Engineer with Spark and ScalaAbout Us"Capco, a Wipro company, is a global technology and management consulting firm. Awarded with

Consultancy of the year in the British Bank Award

and has been ranked

Top Best Companies for Women in India by

. With our presence across 32 cities across globe, we support + clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery.

WHY JOIN CAPCO?You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry.

The projects that will transform the financial services industry.
MAKE AN IMPACTInnovative thinking, delivery excellence and thought leadership to help our clients transform their business.

Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.#BEYOURSELFATWORKCapco has a tolerant, open culture that values diversity, inclusivity, and creativity.

CAREER ADVANCEMENTWith no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.

DIVERSITY & INCLUSIONWe believe that diversity of people and perspective gives us a competitive advantage.
Location-Sarjapur, Bangalore/ Sholinganallur, Chennai - Hybrid

Key Skills:
Data Engineering, Spark, Scala, Airflow, Azure

Technical Requirement:

Role:

Data Engineer / Senior Data EngineerExp:

7+ YearsResponsibilitiesResponsible for designing, building, and maintaining data pipelines and infrastructure on Microsoft AzureExtract, transform, and load data from various sources including databases, APIs, and flat files.

Stay up to date with the latest advancements in Azure data technologies and best practices.
Propose suitable data migration sets to the relevant stakeholders.
Assist teams with processing the data migration sets as required.

Assist with the planning, tracking and coordination of the data migration team and with the migration run book and the scope for each customer.

Role RequirementsStrong Data Analysis experience in Financial ServicesKnowledge of and experience using data models and data dictionaries in a Banking and Financial Markets contextDemonstrate a continual desire to implement "strategic" or "optimal" solutions and where possible, avoid workarounds or short-term tactical solutions.

Working with stakeholders to ensure that negative customer and business impacts are avoided.
Manage stakeholder expectations and ensure that robust communication and escalation mechanisms are in place across the project portfolio.

Good understanding of the control requirement surrounding data handlingExperience/SkillsetProven experience working with Microsoft Azure Data Services, including Data Factory, Data Lake Storage, Synapse Analytics, and Azure SQL Database.

Strong knowledge of data warehousing, data modeling, and ETL/ELT principles.
Hands-on experience in Airflow, ElasticPerform data cleansing, transformation, and integration tasks to ensure data quality and consistency.
Develop and maintain data models for data warehousing, data lakes, and other data storage solutions.

Proficiency in programming languages like Python, PySpark, Scala, or SQL.Knowledge of and experience using data models and data dictionaries in a Banking and Financial Markets contextKnowledge of CI/CD processes for application software integration and deployment, including Maven, Git, and JenkinsImplement and maintain security best practices for data storage and access.

Monitor and troubleshoot data pipelines and data quality issues.
Knowledge of COBOL copy book is preferable to migrate the COBOL copy book solution to Azure data solution.
Document and communicate technical solutions effectively.
Enthusiastic and energetic problem solver to join an ambitious team.
Business analysis skills, defining and understanding requirements.
Attention to detail.

Good knowledge of SDLC and formal Agile processes, a bias towards TDD and a willingness to test products as part of the delivery cycle.

Ability to communicate effectively in a multi-program environment across a range of stakeholders.
Strong verbal and written communication skills

  • Spark/Scala Developer

    2 weeks ago

    Chennai, Tamil Nadu, India RiskInsight Consulting Pvt Ltd Full time

    We are hiring a Data Engineer, and we are looking for someone to join our team. Check out the job details below:Design, develop, and maintain data processing pipelines using Apache Spark and ScalaOptimize Spark jobs for performance, scalability, and reliabilityCollaborate with data engineers and data scientists to implement data-driven solutionsDevelop and...

  • Spark/Scala Developer

    2 months ago

    Chennai, Tamil Nadu, India RiskInsight Consulting Pvt Ltd Full time

    About the job :- Design, develop, and maintain data processing pipelines using Apache Spark and Scala- Optimize Spark jobs for performance, scalability, and reliability- Work closely with data engineers and data scientists to implement data-driven solutions- Develop and maintain ETL processes to ingest, transform, and load large volumes of data- Perform data...

  • Spark/Scala Developer

    4 weeks ago

    Chennai, Tamil Nadu, India RiskInsight Consulting Pvt Ltd Full time

    About the job :- Design, develop, and maintain data processing pipelines using Apache Spark and Scala- Optimize Spark jobs for performance, scalability, and reliability- Work closely with data engineers and data scientists to implement data-driven solutions- Develop and maintain ETL processes to ingest, transform, and load large volumes of data- Perform data...

  • Spark Scala Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India CAPCO Full time

    Spark Scala Data Engineer - Bangalore/ Chennai at Capco India Job Title: Data Engineer with Spark and Scala About Us "Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top Best Companies for Women in India by . With our presence...

  • Big Data- Spark Scala

    2 weeks ago

    Chennai, Tamil Nadu, India Nityo Infotech Full time

    A , Bangalore , Big Data , Data , Level , Level A , Location , Pune , Scala , Spark Experience Required 3 - 9 Years Industry Type IT Employment Type Permanent Location India

  • Python Developer

    4 days ago

    Bangalore/Chennai, Tamil Nadu, India Visionyle Solutions Full time

    Job Title : Python Developer with Spark and ScalaLocation : Bangalore & ChennaiClient : Socit GnraleShift : UK ShiftJob Description :We are looking for a highly skilled Python Developer with expertise in Spark and Scala to join our team for the Socit Gnrale project. The ideal candidate will also possess secondary skills in Data Lake (LUCID), Azure, and...

  • Big Data- Spark Scala

    2 weeks ago

    Chennai, Tamil Nadu, India Nityo Infotech Full time

    A , Bangalore , Big Data , Data , Level , Level A , Location , Pune , Scala , SparkExperience Required 9 YearsIndustry TypeITEmployment TypePermanentLocationIndia

  • Data Engineer

    2 months ago

    Chennai, Tamil Nadu, India m2p fintech Full time

    Location : ChennaiExperience : 3-6 yearsJob Description :What You'll Do :M2P Fintech is looking for Data Engineer to build and set up data solutions. As part of a cross functional team, you will be working in a fast-paced and agile work environment delivering quality and innovative solutions that have immediate business impact.You need to take the...

  • Colan Infotech

    2 weeks ago

    Chennai, Tamil Nadu, India Colan Infotech Pvt Ltd Full time

    Skill Set : Pyspark / Scala Spark, Python, Azure Databricks and Azure Data FactoryRoles And Responsibilities:- Should have programming skills with the ability to write optimized and reusable high-quality code.- Design, develop and maintain scalable data pipelines using Pyspark / Scala Spark, Databricks, Python, SQL.- Optimize SQL queries for efficient data...

  • Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Colan Infotech Private Limited Full time

    Job description Role: Azure Data Engineer Experience: 3-5 Years Skill set: Pyspark / Scala Spark, Python, Azure Databricks and Azure Data Factory Roles and Responsibilities: Should have programming skills with the ability to write optimized and reusable high-quality code. Design, develop and maintain scalable data pipelines using Pyspark / Scala Spark,...

  • Data Engineering

    2 weeks ago

    Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time

    Job Description: We are looking for Data Engineers to join our team. You will use various methods to transform raw data into useful data systems. For example, you'll create algorithms and conduct statistical analysis. Overall, you'll strive for efficiency by aligning data systems with business goals. To succeed in this position, you should have strong...

  • Azure Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Colan Infotech Private Limited Full time

    Job descriptionRole: Azure Data EngineerExperience: 3-5 YearsSkill set:Pyspark / Scala Spark, Databricks, Python, SQL. Must have cloud knowledge of anyone (Azure, AWS, GCP) Roles and Responsibilities: Should have programming skills with the ability to write optimized and reusable high-quality code. Design, develop and maintain scalable data pipelines using...

  • Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time

    JobDescription: Weare looking for Data Engineers to join our team. You will usevarious methods to transform raw data into useful data systems. Forexample you ll create algorithms and conduct statistical analysis.Overall you ll strive for efficiency by aligning data systems withbusiness goals. Tosucceed in thisposition you should have strong analytical skills...

  • Azure Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Colan Infotech Private Limited Full time

    Job description Role: Data Engineer Experience : 3-5 Years Skill set: Pyspark / Scala Spark, Databricks, Python, SQL. Must have cloud knowledge of anyone (Azure, AWS, GCP) Roles and Responsibilities: Should have programming skills with the ability to write optimized and reusable high-quality code. Design, develop and maintain scalable data pipelines...

  • Spark/Scala Developer

    2 weeks ago

    Bangalore/Chennai, Tamil Nadu, India ca-one tech cloud inc Full time

    Job Description :Role : Spark Scala DeveloperExperience : 7+ YearsLocation : Bangalore and ChennaiTechnologies :1. Strong experience with Scala2. Understanding of MongoDB or other NoSQL databases3. Experience using AWS managed services4. Experience with caching architectures, particularly Memcached5. Strong experience with containerization and container...

  • Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Nityo Infotech Full time

    Full Stack Development background with Java and JavaScript/CSS/HTML Knowledge of ReactJs/Angular is a plus Big Data Engineer with solid background with the larger Hadoop ecosystem and real-time analytics tools including PySpark/Scala-Spark/Hive/Hadoop CLI/MapReduce/Storm/Kafka/Lambda Architecture Comfortable with using the larger Hadoop eco system Familiar...

  • AVP C12- Data Engineering

    2 weeks ago

    Chennai, Tamil Nadu, India 12542 Citicorp Services India Private Limited Full time

    Data Engineer:The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.Responsibilities:Responsible for design and development of big data...

  • Azure Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Maneva Consulting Pvt. Ltd Full time

    GreetingsFromManevaJobDescriptionJobTitle Azure Data EngineerLocationChennai CoimbatorePuneExperience6 10YearsShift 12:30 to 9:30 pmJobRequirement:Experience in Azure Data Factory Azure Data bricks Spark (PySparkor scala)SQLExperience in Azure ingestion from onprem source e.g. mainframe SQLserverOracle.Experience in Sqoop /Hadoop.Microsoft Excel (for...

  • Data Engineering

    2 weeks ago

    Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time

    Job Description :We are looking for Data Engineers to join our team. You will use various methods to transform raw data into useful data systems. For example, you'll create algorithms and conduct statistical analysis. Overall, you'll strive for efficiency by aligning data systems with business goals. Tosucceed in this position, you should have strong...

  • Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Blackstraw Full time

    Data EngineerCompany Name:Office Location: Chennai (Work from Office)Job Type: Full-timeExperience: 4+ YearsJob Description: We are looking for Data Engineers to join our team. You will use various methods to transform raw data into useful data systems. For example, you'll create algorithms and conduct statistical analysis. Overall, you'll strive for...