Data engineer

2 weeks ago

Navi Mumbai, India Celebal Technologies Full time

Job Description – ETL Migration Specialist (Informatica to Databricks) Position Overview We are looking for an experienced ETL Migration Specialist with strong expertise in Informatica Power Center and Databricks (Py Spark). The ideal candidate will be responsible for analysing and reverse engineering existing Informatica mappings/workflows, understanding the business logic implemented, and refactoring these pipelines into scalable, efficient data processing workflows on Databricks. This role requires strong analytical skills, hands-on experience in ETL development, and the ability to collaborate with both business and technical teams to ensure seamless migration. ________________________________________ Key Responsibilities • Reverse Engineering: Analyze existing Informatica mappings, workflows, sessions, and variables to understand data flow, transformations, joins, filters, lookups, and aggregations. • ETL Refactoring: Translate Informatica logic into Py Spark/Databricks workflows, ensuring accuracy and performance. • Data Modeling & Transformation: Design and implement transformation logic in Databricks aligned with business rules. • Performance Optimization: Refactor ETL logic for scalability and performance tuning in Databricks. • Documentation: Create detailed documentation for the converted pipelines, capturing both technical logic and business rules. • Testing & Validation: Collaborate with QA/Business teams to validate data correctness between Informatica outputs and Databricks outputs. • Best Practices: Ensure adherence to Databricks and cloud data engineering best practices, including modular coding, error handling, and reusability. • Collaboration: Work with stakeholders, data architects, and business SMEs to clarify requirements and resolve discrepancies. ________________________________________ Must Have Informatica Expertise • Strong knowledge of Informatica objects: o Mappings, Mapplets, Workflows, Worklets, Sessions • Hands-on experience with Informatica transformations: o Lookup, Joiner, Router, Aggregator, Expression, Sequence Generator, Filter, Stored Procedure, Update Strategy, etc. • Proficiency in handling Informatica parameters and variables: o Mapping variables and parameters o Workflow variables o Session parameters o Parameter files and variable scoping • Ability to debug and trace variable values across sessions and workflows. • Experience in designing reusable transformations and optimization techniques in Informatica. ________________________________________ Required Skills & Experience • 5+ years of hands-on experience in ETL development with Informatica Power Center (or IICS). • 2+ years of experience in Databricks (Py Spark, SQL, Delta Lake) for ETL/ELT implementation. • Strong understanding of data warehousing concepts (SCDs, fact/dimension modeling, partitioning, indexing). • Hands-on experience with cloud platforms (Azure / AWS / GCP) and Databricks workspace setup. • Proficiency in SQL and performance tuning of large datasets. • Experience in reverse engineering ETL logic and re-implementing in modern platforms. • Excellent problem-solving, debugging, and communication skills. ________________________________________ Good to Have • Experience with Dev Ops/CI-CD pipelines for Databricks (e.g., Git Hub Actions, Azure Dev Ops, Jenkins). • Exposure to Informatica to Spark migration tools or accelerators. • Knowledge of Medallion architecture or modern data lakehouse patterns. • Experience in Agile/Scrum delivery.

  • Lead Mechanical Engineer

    2 weeks ago

    Navi Mumbai, India BDx Data Centers Full time

    Big Data Exchange (BDX), A leading Pan Asian hybrid, hyperscale and edge solutions provider with assets located in Hong Kong, Singapore, Guangzhou, Nanjing, Indonesia and expanding footprints in SEA regions. Our hybrid ecosystem provides colocation, Enterprise Services, private cloud services, disaster recovery services, and interconnect solutions across...

  • Site Engineer

    14 hours ago

    Navi Mumbai, Maharashtra, India M L Labade Engineer Contractor Full time ₹ 4,00,000 - ₹ 12,00,000 per year

    Role & responsibilitiesOrganizing materials and ensuring sites are safe and clean.Preparing cost estimates and ensuring appropriate materials and tools are available.Providing technical advice and suggestions for improvement on particular projects.Diagnosing and troubleshooting equipment as required.Negotiating with suppliers and vendors to ensure the best...

  • Electrical Design Engineer

    4 days ago

    Navi Mumbai, Maharashtra, India Nxtra Data Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job DescriptionJob Title: Electrical Engineer (Design)Job Level: ManagerFunction: EngineeringLocation : Pan IndiaReporting to : Engineering HeadPurpose of the Job1) Use expertise to assist in the design of electrical systems; helping to avoid problems and addressing any issues that might arise in the reliability of the design, thus reducing downtime and...

  • Electrical engineering lead

    2 weeks ago

    Navi Mumbai, India BDx Data Centers Full time

    Big Data Exchange (BDX), A leading Pan Asian hybrid, hyperscale and edge solutions provider with assets located in Hong Kong, Singapore, Guangzhou, Nanjing, Indonesia and expanding footprints in SEA regions. Our hybrid ecosystem provides colocation, Enterprise Services, private cloud services, disaster recovery services, and interconnect solutions across...

  • Electrical engineering innovation lead

    2 weeks ago

    Navi Mumbai, India BDx Data Centers Full time

    COMPANY OVERVIEW:Big Data Exchange (BDX), A leading Pan Asian hybrid, hyperscale and edge solutions provider with assets located in Hong Kong, Singapore, Guangzhou, Nanjing, Indonesia and expanding footprints in SEA regions. Our hybrid ecosystem provides colocation, Enterprise Services, private cloud services, disaster recovery services, and interconnect...

  • Electrical Engineering Innovation Lead

    2 weeks ago

    Navi Mumbai, India BDx Data Centers Full time

    COMPANY OVERVIEW: Big Data Exchange (BDX), A leading Pan Asian hybrid, hyperscale and edge solutions provider with assets located in Hong Kong, Singapore, Guangzhou, Nanjing, Indonesia and expanding footprints in SEA regions. Our hybrid ecosystem provides colocation, Enterprise Services, private cloud services, disaster recovery services, and interconnect...

  • Electrical engineering innovation lead

    2 weeks ago

    Navi Mumbai, India BDx Data Centers Full time

    COMPANY OVERVIEW:Big Data Exchange (BDX), A leading Pan Asian hybrid, hyperscale and edge solutions provider with assets located in Hong Kong, Singapore, Guangzhou, Nanjing, Indonesia and expanding footprints in SEA regions. Our hybrid ecosystem provides colocation, Enterprise Services, private cloud services, disaster recovery services, and interconnect...

  • Electrical Engineering Innovation Lead

    1 week ago

    Navi Mumbai, India BDx Data Centers Full time

    COMPANY OVERVIEW:Big Data Exchange (BDX), A leading Pan Asian hybrid, hyperscale and edge solutions provider with assets located in Hong Kong, Singapore, Guangzhou, Nanjing, Indonesia and expanding footprints in SEA regions. Our hybrid ecosystem provides colocation, Enterprise Services, private cloud services, disaster recovery services, and interconnect...

  • Electrical Engineering Innovation Lead

    7 days ago

    navi mumbai, India BDx Data Centers Full time

    COMPANY OVERVIEW:Big Data Exchange (BDX), A leading Pan Asian hybrid, hyperscale and edge solutions provider with assets located in Hong Kong, Singapore, Guangzhou, Nanjing, Indonesia and expanding footprints in SEA regions. Our hybrid ecosystem provides colocation, Enterprise Services, private cloud services, disaster recovery services, and interconnect...

  • Electrical engineering innovation lead

    4 weeks ago

    Navi Mumbai, India BDx Data Centers Full time

    COMPANY OVERVIEW: Big Data Exchange (BDX), A leading Pan Asian hybrid, hyperscale and edge solutions provider with assets located in Hong Kong, Singapore, Guangzhou, Nanjing, Indonesia and expanding footprints in SEA regions. Our hybrid ecosystem provides colocation, Enterprise Services, private cloud services, disaster recovery services, and interconnect...