Data Engineer

1 week ago

Gurugram, India Pylon Full time

Description :Responsibilities :- Set up workflows and orchestration processes to streamline data pipelines and ensure efficient data movement within the Azure ecosystem.- Create and configure compute resources within Databricks, including All-Purpose and SQL Compute and Job Clusters to support data processing and analysis.- Set up and manage Azure Data Lake (ADLS) Gen 2 storage accounts and establish a seamless integration with Databricks Workspace for data ingestion and processing.- Create and manage Service Principals, key vaults to securely authenticate and authorize access to Azure resources.- Utilize ETL (Extract, Transform, Load) techniques to design and implement data warehousing solutions and ensure compliance with data governance policies.- Develop highly automated ETL scripts for data processing.- Scale infrastructure resources based on workload requirements, optimizing performance and cost-efficiency.- Profile new data sources in a different format, including CSVs, JSONs, etc.- Apply problem-solving skills to address complex business and technical challenges, such as data quality issues, performance bottlenecks, and system failures.- Demonstrate excellent soft skills and the ability to effectively communicate and collaborate with clients, stakeholders, and cross-functional teams.- Implement Continuous Integration/Continuous Deployment (CI/CD) practices to automate the deployment and testing of data pipelines and infrastructure changes.- Delivering tangible value very rapidly, collaborating with diverse teams of varying backgrounds and disciplines.- Codifying best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases.- Manage timely, appropriate communication and relationships with clients, partners, and other stakeholders.- Create and manage periodic reporting of project execution status and other trackers in standard accepted formats.Requirements :- Exp in the Data Engineering domain : 2+ Years.- Skills : SQL, Python, PySpark. Spark, Distributed Systems. - Azure Databricks, Azure Data Factory, ADLS Gen 2 Blob Storage- Key Vaults, Azure DevOps. - ETL, Building Data Pipelines, Data Warehousing, Data Modelling, and Governance.- Agile Practices, SDLC, Multi-year experience with Azure-Databricks ecosystem and PySpark.- Ability to write clean, concise, and organized PySpark code. - Ability to break down the project into executable steps, prepare a DFD, and execute the same.- Propose innovative DE solutions to achieve business objectives. Quick on his feet, good at tech, and has logically complex communication.- Good Knowledge of ADF, Docker to Have : - Event Hubs, Logic Apps. - Power BI Competitive coding and knows most PySpark syntax by heart. (ref:hirist.tech)

  • Senior Data Engineer

    5 days ago

    Gurugram, India Pacific Data Integrators Full time

    Role: Senior Data Engineer Location: Remote Job Type: Full-time Shift time: Open to work in EST shift (5PM to 2AM IST) Key Responsibilities - Lead the design, development, and implementation of complex data integration solutions using Informatica Intelligent Data Management Cloud (IDMC). - Develop, document, unit test, and maintain high-quality ETL...

  • Senior Data Engineer

    5 days ago

    Gurugram, India Pacific Data Integrators Full time

    Role: Senior Data Engineer Location: Remote Job Type: Full-time Shift time: Open to work in EST shift (5PM to 2AM IST) Key Responsibilities Lead the design, development, and implementation of complex data integration solutions using Informatica Intelligent Data Management Cloud (IDMC). Develop, document, unit test, and maintain high-quality ETL applications...

  • Senior Data Engineer

    4 days ago

    Gurugram, India Pacific Data Integrators Full time

    Role: Senior Data Engineer Location: Remote Job Type: Full-time Shift time: Open to work in EST shift (5PM to 2AM IST) Key Responsibilities Lead the design, development, and implementation of complex data integration solutions using Informatica Intelligent Data Management Cloud (IDMC). Develop, document, unit test, and maintain high-quality ETL applications...

  • Senior Data Engineer

    5 days ago

    Gurugram, India Pacific Data Integrators Full time

    Role: Senior Data EngineerLocation: RemoteJob Type: Full-timeShift time: Open to work in EST shift (5PM to 2AM IST) Key ResponsibilitiesLead the design, development, and implementation of complex data integration solutions using Informatica Intelligent Data Management Cloud (IDMC).Develop, document, unit test, and maintain high-quality ETL applications that...

  • Senior Data Engineer

    6 days ago

    Gurugram, India Pacific Data Integrators Full time

    Role: Senior Data EngineerLocation: RemoteJob Type: Full-timeShift time: Open to work in EST shift (5PM to 2AM IST) Key ResponsibilitiesLead the design, development, and implementation of complex data integration solutions using Informatica Intelligent Data Management Cloud (IDMC).Develop, document, unit test, and maintain high-quality ETL applications that...

  • Senior Data Engineer

    6 days ago

    Gurugram, India Pacific Data Integrators Full time

    Role: Senior Data EngineerLocation: RemoteJob Type: Full-timeShift time: Open to work in EST shift (5PM to 2AM IST) Key ResponsibilitiesLead the design, development, and implementation of complex data integration solutions using Informatica Intelligent Data Management Cloud (IDMC).Develop, document, unit test, and maintain high-quality ETL applications that...

  • Senior Data Consultant

    6 days ago

    Gurugram, India Eucloid Data Solutions Full time

    We are looking for a high-energy individual for the role of Senior Data Consultant at Eucloid. The candidate will collaborate with client stakeholders on multiple projects in the eCommerce team. The candidate is expected to lead following work-streams: • Drive client meetings with data-driven insights and observations on key website/app metricsfor Upper...

  • Senior Data Consultant

    6 days ago

    Gurugram, India Eucloid Data Solutions Full time

    We are looking for a high-energy individual for the role of Senior Data Consultant at Eucloid. The candidate will collaborate with client stakeholders on multiple projects in the eCommerce team. The candidate is expected to lead following work-streams: • Drive client meetings with data-driven insights and observations on key website/app metricsfor Upper...

  • SOC Engineer

    2 weeks ago

    Gurugram, India NTT DATA Full time

    Job Description Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive. Your day at NTT...

  • Digital Engineering Sr. Engineer

    2 weeks ago

    Gurugram, India NTT DATA Full time

    Job Description Req ID:     NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Digital Engineering Sr. Engineer to join our team in Guragon, Delhi (IN-DL), India (IN). Who we are: NTT...