Data Engineer

1 week ago

Gurgaon, Haryana, India Pylon Management Consulting Full time ₹ 8,00,000 - ₹ 24,00,000 per year

Description :

Responsibilities :

- Set up workflows and orchestration processes to streamline data pipelines and ensure efficient data movement within the Azure ecosystem.

- Create and configure compute resources within Databricks, including All-Purpose and SQL Compute and Job Clusters to support data processing and analysis.

- Set up and manage Azure Data Lake (ADLS) Gen 2 storage accounts and establish a seamless integration with Databricks Workspace for data ingestion and processing.

- Create and manage Service Principals, key vaults to securely authenticate and authorize access to Azure resources.

- Utilize ETL (Extract, Transform, Load) techniques to design and implement data warehousing solutions and ensure compliance with data governance policies.

- Develop highly automated ETL scripts for data processing.

- Scale infrastructure resources based on workload requirements, optimizing performance and cost-efficiency.

- Profile new data sources in a different format, including CSVs, JSONs, etc.

- Apply problem-solving skills to address complex business and technical challenges, such as data quality issues, performance bottlenecks, and system failures.

- Demonstrate excellent soft skills and the ability to effectively communicate and collaborate with clients, stakeholders, and cross-functional teams.

- Implement Continuous Integration/Continuous Deployment (CI/CD) practices to automate the deployment and testing of data pipelines and infrastructure changes.

- Delivering tangible value very rapidly, collaborating with diverse teams of varying backgrounds and disciplines.

- Codifying best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases.

- Manage timely, appropriate communication and relationships with clients, partners, and other stakeholders.

- Create and manage periodic reporting of project execution status and other trackers in standard accepted formats.

Requirements :

- Exp in the Data Engineering domain : 2+ Years.

- Skills : SQL, Python, PySpark. Spark, Distributed Systems.

- Azure Databricks, Azure Data Factory, ADLS Gen 2 Blob Storage

- Key Vaults, Azure DevOps.

- ETL, Building Data Pipelines, Data Warehousing, Data Modelling, and Governance.

- Agile Practices, SDLC, Multi-year experience with Azure-Databricks ecosystem and PySpark.

- Ability to write clean, concise, and organized PySpark code.

- Ability to break down the project into executable steps, prepare a DFD, and execute the same.

- Propose innovative DE solutions to achieve business objectives. Quick on his feet, good at tech, and has logically complex communication.

- Good Knowledge of ADF, Docker /containerization.

Good to Have :

- Event Hubs, Logic Apps.

- Power BI Competitive coding and knows most PySpark syntax by heart.

  • Data Engineer

    1 week ago

    Gurgaon, Haryana, India CoPoint Data Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Senior Data Engineer Location: India (Gurugram)About CoPointAIAI isn't coming — it's here. And we help enterprises make it real. At CoPointAI, we work inside the enterprise — not just around it — to turn AI potential into practical wins. From hands-on C-suite workshops (our AI Foundations series) to AI-native MVPs built in weeks, our team partners...

  • Data Engineer Advisor

    1 week ago

    Gurgaon, Haryana, India NTT DATA Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Req ID: 338198NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer Advisor to join our team in Gurgaon, Haryāna (IN-HR), India (IN). Technical Support: Provide L1, L2...

  • Data Scientist

    5 days ago

    Gurgaon, Haryana, India Data Sutram Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    *Data Sutram,*funded by India Infoline (IIFL), Indian Angel Network (IAN) and 100x.VC (led by Sanjay Mehta), is an alternate data company, using external data feeds to create every location's DNA that is utilized in various use cases like credit underwriting, location profiling, site selection, etc. It is one of the fastest-growing companies in the space of...

  • Data Engineer Advisor

    1 week ago

    Gurgaon, Haryana, India NTT DATA North America Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Req ID:338198NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer Advisor to join our team in Gurgaon, Haryāna (IN-HR), India (IN).Technical Support:Provide L1, L2 and L3...

  • Software Engineer

    6 days ago

    Gurgaon, Haryana, India Auto Data Catalog Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Company DescriptionAuto Data Catalog (ADC) is revolutionizing how businesses interact with their data by building an AI-powered platform that eliminates barriers between business users and their data insights. Our AI agents offer instant, actionable insights through simple questions in plain English, transforming data analytics from a technical bottleneck...

  • Data Engineer

    6 days ago

    Gurgaon, Haryana, India NatWest Group Full time ₹ 1,50,00,000 - ₹ 2,50,00,000 per year

    Data Engineer,VP Join us as a Data EngineerYou'll be the voice of our customers, using data to tell their stories and put them at the heart of all decision-making We'll look to you to drive the build of effortless, digital first customer experiences If you're ready for a new challenge and want to make a far-reaching impact through your work, this...

  • Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India NatWest Group Full time US$ 1,50,000 - US$ 2,00,000 per year

    Join us as a Data Engineering LeadThis is an exciting opportunity to use your technical expertise to collaborate with colleagues and build effortless, digital first customer experiencesYou'll be simplifying the bank through developing innovative data driven solutions, inspiring to be commercially successful through insight, and keeping our customers' and the...

  • Data Engineer

    2 days ago

    Gurgaon, Haryana, India Freecharge Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Title : Engineering (Data Engineering)Location :GurgaonExperience Required :1+ YearsEmployment Type :Full-TimeDepartment : Data & AnalyticsAbout The RoleWe are looking for a motivated Data Engineer to contribute to the design and development of scalable data frameworks and solutions. The role involves working with big data technologies like Spark, Kafka,...

  • Data Engineer

    6 days ago

    Gurgaon, Haryana, India NatWest Group Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Data Engineer, AVP Join us as a Data EngineerWe're looking for someone to build effortless, digital first customer experiences to help simplify our organisation and keep our data safe and secure Day-to-day, you'll develop innovative, data-driven solutions through data pipelines, modelling and ETL design while inspiring to be commercially successful...

  • Data Engineer

    3 days ago

    Gurgaon, Haryana, India Terra Technology Circle Consulting Private Limited Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Title:Data EngineerExperience:4–8 yearsLocation:GurugramEmployment Type:ContractualAbout the RoleWe are seeking a highly skilled and motivatedData Engineerto join our growing data and analytics team. In this role, you will design, build, and optimize scalable data pipelines and architectures to support analytics, machine learning, and business...