Data Engineer

1 week ago


Ahmedabad, Gujarat, India Talentoj Full time

Azure Data Engineer :


We are seeking a highly skilled Azure Data Engineer to join the data engineering team and play a crucial role in an optimization initiative.

The current problem is that read speed is compromised and faster responses from delta are required.

The ideal candidate should have extensive experience in Python programming and be proficient in PySpark and Databricks. The selected candidate will be responsible for designing and implementing efficient data pipelines and solutions for data-driven projects in Azure Data Factory and Azure Databricks

Responsibilities :

  • Design, build, and maintain scalable data pipelines using PySpark and Databricks
  • Optimize data processing and storage for maximum performance and efficiency
  • Troubleshoot and debug datarelated issues, and implement solutions to prevent reoccurrence
  • Collaborate with data scientists, software engineers, and other stakeholders to ensure that data solutions are aligned with business goals

Requirements :

  • Strong experience in Python programming and PySpark, and SparkSQL
  • Clear understanding of Spark Data structures, RDD, Dataframe, dataset
  • Expertise in Databricks and ADLS
  • Expertise handling data type, from dictionaries, lists, tuples, sets, arrays, pandas dataframes, and spark dataframes
  • Expertise working with complex data types such as, structs, and JSON strings.
  • Clear understanding of Spark Broadcast, Repartition, Bloom index filters
  • Experience with ADLS optimization, partitioning, shuffling and shrinking
  • Ideal experience with disk caching
  • Ideal Experience with cost based optimizer
  • Experience with data modeling, data warehousing, datalake, deltalake and ETL/ELT processes in ADF
  • Strong analytical and problemsolving skills
  • Excellent documentation, communication and collaboration skills
  • Experience of Leading team of engineers and have mentored them
)

  • Ahmedabad, Gujarat, India Crest Data Full time

    Company Overview: Crest Data is the global leading provider of Data Analytics, Security, DevOps, Cloud Solutions, Software integrations, Analytics, and security-based technological services. With a clientele that includes several Fortune 500 corporations and some of the innovative Silicon Valley Startups. Company URL : Job Location - Ahmedabad & Pune Key...

  • MS Engineer

    1 week ago


    Ahmedabad, Gujarat, India NTT DATA Full time

    Job Description NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future. In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our...

  • Data Engineer

    1 week ago


    Ahmedabad, Gujarat, India Kraft Heinz Full time

    Data Engineer: Work from Office: AhmedabadDescription & Requirements:About the PositionWe have an excellent opportunity available for an Associate Data Engineer to make a difference at Kraft Heinz. The Associate Data Engineer role lies within the newly created Analytics Centre of Excellence within the Global Business Services organization. The Analytics COE...

  • Data Engineer

    1 week ago


    Ahmedabad, Gujarat, India Kraft Heinz Full time

    Data Engineer: Work from Office: Ahmedabad Description & Requirements: About the Position We have an excellent opportunity available for an Associate Data Engineer to make a difference at Kraft Heinz.The Associate Data Engineer role lies within the newly created Analytics Centre of Excellence within the Global Business Services organization.The Analytics COE...

  • Data Engineer

    1 week ago


    Ahmedabad, Gujarat, India Sterlite Technologies Limited Full time

    Role as a Data engineer :Develop and build AI/ML model ready data ponds using Azure Databricks Delta lake Improve upon the data ingestion models, ETL jobs, and alarm to maintain data integrity and data availability. Partner with AI/ML product teams proactively identify the data needs for rapid experimentation, stable production deployment and ensure the...

  • Data Engineer

    1 week ago


    Ahmedabad, Gujarat, India Kraft Heinz Full time

    Data Engineer: Work from Office: AhmedabadDescription & Requirements:About the PositionWe have an excellent opportunity available for an Associate Data Engineer to make a difference at Kraft Heinz. The Associate Data Engineer role lies within the newly created Analytics Centre of Excellence within the Global Business Services organization. The Analytics COE...

  • Data Engineer

    1 week ago


    Ahmedabad, Gujarat, India Paradise Placement Consultancy Full time

    Responsibilities development of methods and tools for data validation and analysis and in planning and execution of complex projects Focus on Quality Analytics (e.g., Mean Time between Failures, Mean Time to Crash, Mean Time to Hang, and other quality metrics) Troubleshoot complex, high risk data issues and perform root cause analysis to resolve...

  • Data Engineer

    1 week ago


    Ahmedabad, Gujarat, India TIGI HR ® Full time

    Position- Data Engineer Experience- 4-6 Years Budget- 9.00LPA Location - Ahmedabad Notice period- Immediate - 30 Days Roles & Responsibilities: Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights.  Constructing infrastructure (on cloud - GCP) for efficient ETL processes...

  • Data Engineer

    1 week ago


    Ahmedabad, Gujarat, India Acesoft Labs (India) Pvt Ltd Full time

    We are looking for a talented Data Engineer to join our team at XYZ Company!Responsibilities:Build efficient data pipelines for training examples and online trainingDesign and implement data transformations for various purposesCreate scalable big data systems to support our core productsIdentify opportunities for application improvementsManage project...


  • Ahmedabad, Gujarat, India Ridgeant Techonologies Full time

    Job Description : AWS Data EngineerResponsibilities : Collaborate with stakeholders to understand business requirements and data needs, and translate them into scalable and efficient data engineering solutions using AWS Data Services. Design, develop, and maintain data pipelines using AWS serverless technologies such as Glue, S3, Lambda, DynamoDB, Athena,...

  • Data Architect

    1 month ago


    Ahmedabad, Gujarat, India Headsup Corporation Pvt. Ltd. Full time

    As a Data Architect, you will be the driving force behind our data strategy. You will be responsible for designing and implementing a robust and scalable data architecture, collaborating with cross-functional teams, and ensuring that our data infrastructure supports the company's growth and analytical needs.Skills : 1. Proven experience in data...

  • Data Architect

    3 weeks ago


    Ahmedabad, Gujarat, India Headsup Corporation Pvt. Ltd. Full time

    As a Data Architect, you will be the driving force behind our data strategy. You will be responsible for designing and implementing a robust and scalable data architecture, collaborating with cross-functional teams, and ensuring that our data infrastructure supports the company's growth and analytical needs.Skills : 1. Proven experience in data...

  • Data Engineer

    1 week ago


    Ahmedabad, Gujarat, India Paradise Placement Consultancy Full time

    Responsibilitiesdevelopment of methods and tools for data validation and analysis and in planning and execution of complex projectsFocus on Quality Analytics (e.g., Mean Time between Failures, Mean Time to Crash, Mean Time to Hang, and other quality metrics)Troubleshoot complex, high risk data issues and perform root cause analysis to resolve issuesIdentify,...


  • Ahmedabad, Gujarat, India YORK IE Ahmedabad Full time

    Job DescriptionWe are looking for a skilled Data Engineer with expertise in Python and a strong understanding of messaging services and serialization frameworks such as Kafka, Pub/Sub, Protobuf, or Avro. The ideal candidate will have experience in building and managing data pipelines, ensuring efficient data flow, and maintaining data integrity across...

  • Data Engineer

    1 week ago


    Ahmedabad, Gujarat, India Techno Wise Full time

    Our US product based client is looking for fewData Engineersfor their Analytics department.Roles:Understand design/mapping documents and implement into code using ETL tool/technology preferred by company.Develop code that adhere to development standards and are accurate, efficient and maintainable.Maintain documentation for all developed ETL processes,...


  • Ahmedabad, Gujarat, India Arrow Electronics Full time

    Position:AWS Data Engineer/Senior EngineerJob Description:Responsibilities: Collaborate with stakeholders to understand business requirements and data needs, and translate them into scalable and efficient data engineering solutions using AWS Data Services.Design, develop, and maintain data pipelines using AWS serverless technologies such as Glue, S3, Lambda,...


  • Ahmedabad, Gujarat, India Arrow Electronics Full time

    Position:AWS Data Engineer/Senior EngineerJob Description:Responsibilities: Collaborate with stakeholders to understand business requirements and data needs, and translate them into scalable and efficient data engineering solutions using AWS Data Services.Design, develop, and maintain data pipelines using AWS serverless technologies such as Glue, S3, Lambda,...


  • Ahmedabad, Gujarat, India Paradise Placement Consultancy Full time

    ResponsibilitiesGuide the Data Engineering team in development of methods and tools for data validation and analysis and in planning and execution of complex projectsFocus on Quality Analytics (e.g., Mean Time between Failures, Mean Time to Crash, Mean Time to Hang, and other quality metrics)Troubleshoot complex, high risk data issues and perform root cause...

  • Data Engineer

    1 week ago


    Ahmedabad, Gujarat, India Techno Wise Full time

    Our US product based client is looking for few Data Engineers for their Analytics department. Roles: Understand design/mapping documents and implement into code using ETL tool/technology preferred by company. Develop code that adhere to development standards and are accurate, efficient and maintainable. Maintain documentation for all developed...

  • Data Engineer

    1 week ago


    Ahmedabad, Gujarat, India GetOnData Solutions Full time

    About the Role : Design and development with Informatica IICS or PowerCenter in either/both Data Warehousing and Enterprise Integration projects Development of ETL (Extract Transform & Load) by utilizing relevant technologies (e.g. Informatica, MS Azure Data Factory, Azure Data Lake,etc.). Develop Data Ingestion and Integration from data sources to Azure...