Ab Initio to Spark Migration Developer

10 hours ago

Pune, India EXL Full time
Position - Ab Initio to Spark Migration Expert
Location - Pune/Bangalore/Gurgaon (Hybrid)
Experience - 4+
We are seeking a skilled Ab Initio to Spark Migration Developer to facilitate our transition from Ab Initio ETL processes to a Spark-based framework. This critical role involves analyzing existing workflows, designing scalable data architectures, and developing high-performance applications using Spark and Scala. You will collaborate closely with cross-functional teams to ensure data solutions meet business needs while also optimizing performance and automating processes. In addition to technical migration, you'll mentor junior team members and promote best practices. Your ability to communicate complex concepts to non-technical stakeholders will be essential for driving successful outcomes in our data initiatives.
Key Responsibilities:
Migration : Transition existing ETL processes from Ab Initio to Spark, ensuring a smooth process with minimal disruption to operations.
Architecture Design : Collaborate with data architects to design scalable and efficient data processing architectures that meet business requirements.
Data Processing Development : Develop high-performance applications using Spark and Scala, implementing best practices in coding and architecture.
SQL Proficiency : Utilize strong SQL skills to query and manipulate data stored in Oracle databases as part of the migration and development process.
Performance Optimization : Identify and implement strategies for optimizing Spark applications to enhance performance and resource utilization.
Automation : Create and maintain shell scripts to automate various ETL tasks and streamline workflows.
Workflow Orchestration : Use Apache Airflow to schedule, monitor, and manage data pipelines, ensuring reliable data flow.
Debugging and Troubleshooting : Proactively debug and resolve issues in Spark applications, employing analytical skills to identify and fix problems effectively.
Documentation : Maintain comprehensive documentation of processes, workflows, and configurations for future reference and knowledge sharing.
Collaboration : Work closely with cross-functional teams, including data scientists, analysts, and business stakeholders, to align on data requirements and project objectives.
Continuous Improvement : Stay informed on industry trends, tools, and best practices related to ETL, big data, and cloud technologies, and recommend enhancements as needed.
Must-Have Skills:
Ab Initio ETL : Extensive experience with Ab Initio, including deep understanding of XFR (Components, Graphs, and Plans).
Spark & Scala : Strong proficiency in developing applications using Spark and Scala is mandatory.
SQL & Oracle : Solid experience in SQL and familiarity with Oracle databases, including data querying and manipulation.
Performance Optimization : Proven ability to analyze and optimize Spark applications for performance, including resource management.
Shell Scripting : Strong skills in writing and managing shell scripts for automation and process management.
Airflow : Proficiency in Apache Airflow for orchestrating complex data workflows and ensuring reliability.
Debugging Skills : Excellent debugging and problem-solving skills to troubleshoot and resolve issues promptly.
Version Control : Experience with GIT and Jenkins for version control and continuous integration processes.
Good-to-Have Skills:
Cloudera Data Platform : Understanding of Cloudera Data Platform and its integration with big data technologies.
Cloud Platforms : Familiarity with cloud data services such as AWS, Azure, or GCP, including tools like AWS Glue or Azure Data Factory.
Data Warehousing : Knowledge of data warehousing concepts and methodologies.
Big Data Technologies : Understanding of other big data technologies (e.g., Hadoop, Hive, Kafka).
Data Modeling : Experience in data modeling and database design, especially in big data contexts.
Agile Methodologies : Previous experience working in Agile/Scrum environments.
Qualifications:
Bachelor’s degree in Computer Science, Information Technology, or a related field.
5-7 years of experience in data engineering or ETL development, with a focus on migration projects.
Proven track record of successful migration projects from Ab Initio to Spark using Scala.

  • Ab Initio to Spark Migration Developer

    9 hours ago

    pune, India EXL Full time

    Position - Ab Initio to Spark Migration Expert Location - Pune/Bangalore/Gurgaon (Hybrid) Experience - 4+ We are seeking a skilled Ab Initio to Spark Migration Developer to facilitate our transition from Ab Initio ETL processes to a Spark-based framework. This critical role involves analyzing existing workflows, designing scalable data architectures, and...

  • Ab Initio to Spark Migration Developer

    15 hours ago

    Pune, India EXL Full time

    Position - Ab Initio to Spark Migration Expert Location - Pune/Bangalore/Gurgaon (Hybrid) Experience - 4+ We are seeking a skilled Ab Initio to Spark Migration Developer to facilitate our transition from Ab Initio ETL processes to a Spark-based framework. This critical role involves analyzing existing workflows, designing scalable data architectures, and...

  • Ab Initio to Spark Migration Developer

    21 hours ago

    Pune, India EXL Full time

    Position - Ab Initio to Spark Migration ExpertLocation - Pune/Bangalore/Gurgaon (Hybrid)Experience - 4+We are seeking a skilled Ab Initio to Spark Migration Developer to facilitate our transition from Ab Initio ETL processes to a Spark-based framework. This critical role involves analyzing existing workflows, designing scalable data architectures, and...

  • Ab Initio to Spark Migration Developer

    7 hours ago

    pune, India EXL Full time

    Position - Ab Initio to Spark Migration ExpertLocation - Pune/Bangalore/Gurgaon (Hybrid)Experience - 4+We are seeking a skilled Ab Initio to Spark Migration Developer to facilitate our transition from Ab Initio ETL processes to a Spark-based framework. This critical role involves analyzing existing workflows, designing scalable data architectures, and...

  • Ab Initio Developer-2

    4 months ago

    Pune, India TransUnion Full time

    TransUnion's Job Applicant Privacy Notice **What We'll Bring**: - 5+ years of hands-on development experience with Ab Initio, Linux, SQL and relational databases. **What You'll Bring**: - 5+ years of hands-on development experience with Ab Initio, Linux, SQL and relational databases. - Ability to demonstrate knowledge of advanced Ab Initio concepts such...

  • Ab Initio

    4 weeks ago

    Pune, India 2coms Full time

    Job DescriptionLooking For AbInitio Development Professionals For Global IT MNC Greetings From 2COMS Group!Experience: 6+ yrsWork Location: Pune (WFO) Project Role: Application Developer Project Role Job Description: Design, build, and configure applications to meet business process and application requirements. Must have Skills: AbInitio Good to Have...

  • Ab Initio

    4 weeks ago

    Pune, India 2coms Full time

    Looking For AbInitio Development Professionals For Global IT MNC Greetings From 2COMS Group!Experience: 6+ yrsWork Location: Pune (WFO) Project Role: Application Developer Project Role Job Description: Design, build, and configure applications to meet business process and application requirements. Must have Skills: AbInitio Good to Have Skills: UNIX Shell...

  • Ab Initio Cloud COE Administrator

    4 months ago

    Pune, India Exusia Full time

    About The Position Department: Sales and Delivery Team - Empower Industry: Information Technology & Services, Computer Software, Management Consulting Location: Anywhere in India Experience Range : 5 to 7 years Basic Qualification: Master/Bachelor of Engineering or Equivalent Travel Requirements: No Website : Role Overview This...

  • Ab Initio Technical Lead PUN

    4 months ago

    pune, India Exusia Full time

    The Ab Initio Technical Lead will be responsible for working on-site with our leading on-site consultants and off-site teams as a hands-on designer and developer of complex components working on strategic implementation projects using the latest technologies from Ab Initio corp. (AI) THIS IS NOT A PROD SUPPORT POSITION. (hands-on practitioners only to apply)...

  • Ab Initio Technical Leads

    2 weeks ago

    pune, India Exusia Full time

    Department:  Sales and Delivery Team - Empower Industry:  Information Technology & Services, Computer Software, Management Consulting Location:  Anywhere in India or United States (US - H1B Visa holder required). If you do not have an existing H1-B visa, please apply to one of our other job posts for domestic work in India. Experience Range : 5 to 12...

  • Ab initio Developer

    3 months ago

    Pune, India Wipro Full time

    Role Purpose The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters Do Instrumental in understanding the requirements and design of the product/ software Develop software solutions by studying information...

  • Abinitio Developer

    5 days ago

    Pune, India 2coms Full time

    Looking For Ab Initio development Professionals For Global IT MNC Greetings From 2COMS Group!Location Gurgaon/Mumbai/Pune/Bangalore/ChennaiExperience (412 years )MustHave: Minimum of 4 years of handson experience in Ab Initio development.Job Description:Minimum of 4 years of handson experience in Ab Initio development. Develop and optimize ETL workflows...

  • Abinitio Developer

    2 weeks ago

    Pune, India 2coms Full time

    Looking ForAbInitio development Professionals For Global IT MNCGreetingsFrom 2COMSGroup!LocationGurgaon/Mumbai/Pune/Bangalore/ChennaiExperience(412 years)MustHave:Minimumof 4 years of handson experience in Ab Initiodevelopment.JobDescription:Minimumof 4 years of handson experience in Ab Initio development.Developand optimize ETL workflows and processes for...

  • Abinitio Developer

    4 months ago

    Pune, India HuQuo Consulting Pvt. Ltd. Full time

    Abinitio Developer :Experience - 3-12 yearsJob Description :- 3-10 years hands on strong experience in Ab-Initio technology.- Should have good knowledge about Ab-Initio components like Reformat, Join, Sort, Rollup, Normalize, Scan, Lookup, MFS, Ab-Initio parallelism and products like Metadata HUB, Conduct>IT, Express>IT, Control center and good to have clear...

  • Abinito Architect

    3 weeks ago

    Pune, India Qualite Manpower Full time

    Role : Ab-initio architectExperience : 10+ YrsLocation : Bangalore/PuneNotice period : Immediate to 30 days /Serving Notice period Responsibilities :1. Requirements Analysis : Collaborate with business stakeholders to understand data integration requirements and translate them into technical solutions using Ab Initio. 2. Solution Design : Design end-to-end...

  • Abinitio Architect

    2 months ago

    Pune, India Virtusa Full time

    Abinitio Architect - CREQ194575 Description Establish and implement new or revised application systems and programs in coordination with the Technology team. Perform applications systems analysis and programming activities. Develop comprehensive knowledge of how areas of business integrate to accomplish business goals. Ensure essential Application...

  • AbInitio/Metadata Hub Engineer

    2 weeks ago

    Pune, India Talpro Full time

    Job Description : Responsibilities :Expertise in Ab Initio Metadata Hub and ETL (Extract, Transform, Load) processes.Prior experience in data lineage projects and writing metadata hub scripts.Knowledge of backend database for Metadata Hub and ability to obtain relevant information.Break down complex problems and implement timely and effective...

  • Abinitio Architect

    4 months ago

    Pune, Maharashtra, India Virtusa Full time

    Review and analyze proposed technical solutions for projects. Develop comprehensive knowledge of how areas of business integrate to accomplish business goals. Ensure essential Application Development procedures are followed and contribute to defining standards negotiating with external parties when necessary. Appropriately assess risk when business decisions...

  • Abinitio Developer

    3 months ago

    pune, India Shavish HR Full time

    Job Description :You will be the part of data migration team leading the ETL code changes for the database. Responsible for the understanding the existing A>I graphs, reconfigure the same for new target (Snowflake), UT, SIT, deploy the modified code. Key Responsibilities :- Understanding the existing A>I graphs- Reconfigure the A>I for new target...

  • Abinitio

    4 months ago

    Pune, India Ashra Technology Full time

    Greetings of the day We are hiring... Title: Abinitio developer Years of exp.: 4 years Location: Pune,Chennai JD: - **Abinitio**: - Should have Abinitio development experience for more than 4+ years - Strong Skills in Data warehousing, Business Intelligence, ETL - Should have good working experience in Ab Initio components such as Continuous...