ETL Data Engineer

3 weeks ago


GurgaonGurugram, IN Coders Brain Pvt Ltd Full time

Job Description: ETL - Data Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.

- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness

- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.

- Diagnose system performance issue related to data processing and implement solution to address them.

- Collaborate with other teams to ensure successful integration of data pipelines into larger system architecture requirement.

- Maintain integrity and quality across all pipelines and environments.

- Understand and follow secure coding practice to make sure code is not vulnerable

Skills:

- Should have Strong background in Software development with experience in ingest, transform and store data from large datasets using Pyspark in Azure Databricks with strong knowledge on distributed computing concepts.

- Must have hands on experience in designing and developing ETL Pipelines in Pyspark in Azure Databricks with strong python scripting exposure like list comprehensions, Dictionary variables etc.

- Must have minimum 5 years of exposure and good proficiency in data warehousing concepts.

- Proficient in SQl and database Design concepts.

- Hands on experience with good proficiency in Delta table and delta file operations like merge, Insert override, Partition overrides etc.

- Hands on Experience in CICD in Azure DevOps/Harness and ADF/Stone branch for orchestration.

- Knowledge of Azure cloud computing platform with Azure Synapse, ADLS.

- Knowledge on GitHub and build management.

- Passion for data and experience working within a data driven organization

- Excellent presentation, communication (oral & written), and relationship building skills, across all levels of management

(ref:hirist.tech)
  • Data Engineer

    3 weeks ago


    Gurgaon/Gurugram, IN HuQuo Consulting Pvt. Ltd. Full time

    Job Description :Role Overview :We are looking for a go-getter person who can define :- Data strategies to meet the demands of business requirements- Technical requirements of the organizational data needs from diverse teams- Data architecture, models and go-forward data solutions for various teams of the organization- The person will be part of the Data...


  • Gurgaon/Gurugram, IN Bigfoot Retail Solutions Pvt. Ltd Full time

    Job Description :- Design and build data pipelines from various cloud data sources for the enterprise data warehouse- Partner with Data Engineers, Data architects, domain experts, data analysts and other teams to build foundational data sets that are trusted, well understood, aligned with business strategy and enable self-service- Own and document data...


  • Gurgaon/Gurugram, IN Huquo Full time

    Job Description :- Expertise on SQL, Python, ETL tools, AWS or other cloud- Ability to improve existing ETL pipelines, automate ETL pipelines, and analytical workflows- Experience in the extraction and manipulation of data to conduct analysis to develop understanding on multiple data systems (BI analytics, data blending, data warehousing, data ETL, etc.)- A...

  • Data Engineer

    3 weeks ago


    Gurgaon/Gurugram, IN HuQuo Consulting Pvt. Ltd. Full time

    Key Responsibilities & Skillsets :Common Skillsets :- 5+ years of experience in analytics, Oracle, ETL, Python and associated data engineering jobs.- Must have experience with managing and transforming big data sets using pyspark, spark-scala- Experience with AWS services - Glue, Lambda, EMR, Redshift, RDS- Excellent communication & presentation skills -...


  • Gurgaon/Gurugram, IN Saarthee Full time

    Position Summary : In the Data Engineer role, you will assist on data retrieval from multiple source systems, data processing and cleansing. You will be responsible for constructing automated data pipelines and updating the serving layer for analytics and insights. Your Role Responsibilities and Duties : - Monitor and troubleshoot operational or data issues...


  • Gurgaon/Gurugram, IN CRESCENDO GLOBAL LEADERSHIP HIRING INDIA PRIVATE L Full time

    Role : Analytics and Data Engineer - SQL| Python| ETL, 6+ years, GurgaonWe are seeking an experienced Analytics & Data Engineering professional to join our team and spearhead data initiatives. As a key player in our organization, you will collaborate directly with client leadership to shape the vision and roadmap of their Data, Analytics & Insights team. If...

  • Data Engineer

    3 weeks ago


    Gurgaon/Gurugram, IN Huquo Consulting Full time

    Role : Data EngineerLocation : GurgaonDesignation : AM- AVP (as per interview performance & EXL fitment)Work Timings : 1pm- 10pmRole Overview :We are looking for a go getter person who can define :1. Analytic strategies to meet the demands of business requirements2. Technical requirements of the analytic solutions3. Data requirements of the analytic solution...

  • Data Engineer

    1 week ago


    Bangalore/Pune/Kolkata/Gurgaon/Gurugram/Mumbai/Ahmedabad, IN Sterlite Technologies Limited Full time

    Role as a Data engineer :- Develop and build AI/ML model ready data ponds using Azure Databricks Delta lake- Improve upon the data ingestion models, ETL jobs, and alarm to maintain data integrity and data availability.- Partner with AI/ML product teams proactively identify the data needs for rapid experimentation, stable production deployment and ensure the...


  • Gurgaon/Gurugram, IN CASHGRAIL PRIVATE LIMITED Full time

    Role & Responsibilities (KRAs) 1) Understand, implement and automate ETL and data pipelines with up-to-date industry standards 2) Hands-on involvement in the design, development and implementation of optimal and scalable data systems 3) Design, build and scale data projects independently and mentor junior data engineers Must - Have Skills: 1) Data Structure...

  • Azure Data Engineer

    3 weeks ago


    Gurgaon/Gurugram, IN Huquo Full time

    Job Description :- To collaborate with various teams/regions in driving facilitating data design, identifying architectural risks and key areas of improvement in data landscape, and developing and refining data models and architecture frameworks- Technical experience and knowledge in Cloud Data Warehousing, data migration and data transformation- Develop and...


  • Gurgaon/Gurugram, IN Intellismith pvt Ltd Full time

    About Intellismith :Intellismith, founded in 2019, is a dynamic HR service and technology startup. Our mission is to tackle India's employability challenges head-on. Currently, we operate two key lines of business: recruiting and outsourcing. With teams based in Noida, Chennai, Mumbai, and Bangalore, we collaborate with top brands in the BFSI and IT...

  • Azure Data Engineer

    2 weeks ago


    Bangalore/Pune/Kolkata/Chennai/Gurgaon/Gurugram, IN VIDPRO CONSULTANCY SERVICES Full time

    Primary Roles and Responsibilities:- Developing Modern Data Warehouse solutions using Databricks and Azure Stack - Ability to provide solutions that are forward-thinking in data engineering and analytics space- Collaborate with DW/BI leads to understand new ETL pipeline development requirements.- Triage issues to find gaps in existing pipelines and fix the...

  • Data Engineer

    3 weeks ago


    Gurgaon/Gurugram/Bangalore, IN Huquo Full time

    Role and Responsibilities :- Liaise with different client stakeholders on ad-hoc analyses related to monitoring the entire data lake- Build ETL & data pipelines to help feed the data into different business facing data products/dashboards- Explore options to automate processes & workflows and thus drive efficiencies for client- Work on ML Model based...

  • ETL Developer

    3 weeks ago


    Gurgaon/Gurugram, IN Sureminds Solutions Pvt. Ltd. Full time

    Job Description :1) Have a strong technical background, including 8+ years of experience in designing and engineering large scale systems.2) At least 3 years of experience working as lead in designing, development and deploying data integration and ETL solutions in a data warehousing and BI environment involving data transformations, change data capture, and...

  • Azure Data Engineer

    2 weeks ago


    Gurgaon/Gurugram, IN Conneqtcorp Full time

    Role - Azure Data EngineerLocation - GurugramWork Mode - HybridSkills - Databricks, Azure, Python, SQLNotice Period - May JoinersRole Overview :We are seeking a skilled Data Engineer to join our team. The successful candidate will be responsible for maintaining and optimizing data pipelines, implementing robust data checks, and ensuring the accuracy and...

  • Big Data Developer

    3 weeks ago


    Gurgaon/Gurugram/Bangalore, IN Workassist.in Full time

    Our hiring partner is looking for :- This is a full-time on-site role for a Big Data Developer at Big Four Consulting Firm Gurugram.- The Big Data Developer will be responsible for tasks related to Data Engineering, Extract Transform Load (ETL), Big Data, Data Warehousing, and Software Data Engineering, Big Data, Extract Transform Load (ETL), and Data...


  • Gurgaon/Gurugram, IN Huquo Full time

    Job Description :- Analytic strategies to meet the demands of business requirements- Technical requirements of the analytic solutions- Data requirements of the analytic solution processes- The person will be part of the Data Science team for a major Insurance client. He/ She will work with different stakeholders as SME for data engineering- Engages in...

  • Data Analyst

    3 weeks ago


    Gurgaon/Gurugram, IN Www.Huquo.com Full time

    Job Description :- Participate in the end-to-end delivery lifecycle of data warehousing projects from business requirements gathering to solution design, development, testing and roll-out.- Support the data development lifecycle activities, including data profiling, ETL and data model design and development, testing, and support.- Leverage data analysis...

  • Senior Data Engineer

    3 weeks ago


    Hyderabad/Chennai/Pune/Mumbai/Gurgaon/Gurugram, IN Hub Work Full time

    We are looking for a data engineer who has experience in building data pipelines for large data platforms. This role will be working with a team building a large data platform catering to the needs of a wealth manager for their application and analytics needs. This role will be augmented with a customer of Intellect and other third parties in delivering...

  • Data Scientist

    3 weeks ago


    Gurgaon/Gurugram, IN ARAHAS TECHNOLOGIES PRIVATE LIMITED Full time

    We are seeking a skilled Data Scientist with 4-6 years of experience to join our team. The ideal candidate should possess strong analytical skills, a solid understanding of statistical techniques, and hands-on experience with data visualization tools like PowerBI. Additionally, expertise in ETL (Extract, Transform, Load) processes is...