Data Engineer

1 month ago

KarnatakaBangarapet, India RiskInsight Consulting Pvt Ltd Full time

About the Role :

We are seeking a skilled and experienced Big Data Engineer to join our team and play a crucial role in building and maintaining our big data infrastructure.

You will leverage your expertise in the Hadoop ecosystem and data processing frameworks to design, implement, and optimize scalable data pipelines.

Responsibilities :

- Design, develop, and maintain efficient data pipelines for large-scale data processing.

- Utilize your proficiency in HDFS, HiveQL, and PySpark to create and optimize data processing workflows.

- Perform data wrangling and transformation tasks to ensure data quality and consistency.

- Write efficient SQL queries for data manipulation and retrieval.

- Automate tasks and schedule jobs using Linux shell scripting.

- Develop and maintain Python scripts for data manipulation, analysis, and automation tasks.

- Collaborate with data analysts and data scientists to understand data needs and translate them into technical solutions.

- Troubleshoot and debug big data processing issues to ensure smooth system operation.

- Continuously monitor and optimize data pipelines for performance and scalability.

- Advocate for best practices in data management and big data technologies.

- Stay up-to-date on the latest trends and innovations in the big data and analytics landscape.

Qualifications :

- 4-8 years of experience as a Data Engineer or a similar role.

- Strong expertise in Hadoop ecosystem components, including HDFS, HiveQL, and YARN.

- Proven experience in developing and optimizing data processing workflows using PySpark.

- Proficiency in SQL for data manipulation and querying.

- Hands-on experience with Linux shell scripting for automation and job scheduling.

- Solid understanding of Python programming language and libraries for data manipulation and analysis.

- Experience with version control systems (Git) is a plus.

- Knowledge of CI/CD pipelines is a plus.

- Excellent communication and collaboration skills to work effectively with cross-functional teams.

- Strong problem-solving skills and a keen eye for detail.

- Proactive and eager to learn new technologies and stay updated in the big data field.

(ref:hirist.tech)

  • Data Engineer

    1 month ago

    Karnataka/Bangarapet, IN RiskInsight Consulting Pvt Ltd Full time

    About the Role :We are seeking a skilled and experienced Big Data Engineer to join our team and play a crucial role in building and maintaining our big data infrastructure. You will leverage your expertise in the Hadoop ecosystem and data processing frameworks to design, implement, and optimize scalable data pipelines.Responsibilities : - Design, develop,...

  • Data Engineer

    4 weeks ago

    Karnataka/Bangarapet, IN RiskInsight Consulting Pvt Ltd Full time

    About the Role :We are seeking a skilled and experienced Big Data Engineer to join our team and play a crucial role in building and maintaining our big data infrastructure. You will leverage your expertise in the Hadoop ecosystem and data processing frameworks to design, implement, and optimize scalable data pipelines.Responsibilities : - Design, develop,...

  • Networking Advisor

    2 weeks ago

    Bangarapet, India NTT DATA Full time

    **Req ID**:223622 We are currently seeking a Networking Advisor to join our team in Delhi, Uttar Pradesh (IN-UP), India (IN). **Position overview** The position requires, day to day operational support for requests and incidents on all network related aspects involving multiple technologies such as: wireless, switching, routing, sdwan, and high level...

  • Data Engineer

    1 month ago

    Bangarapet, India RiskInsight Consulting Pvt Ltd Full time

    About the Role :We are seeking a skilled and experienced Big Data Engineer to join our team and play a crucial role in building and maintaining our big data infrastructure. You will leverage your expertise in the Hadoop ecosystem and data processing frameworks to design, implement, and optimize scalable data pipelines.Responsibilities : - Design, develop,...

  • Data Engineer

    3 weeks ago

    Bangarapet, India RiskInsight Consulting Pvt Ltd Full time

    About the Role :We are seeking a skilled and experienced Big Data Engineer to join our team and play a crucial role in building and maintaining our big data infrastructure. You will leverage your expertise in the Hadoop ecosystem and data processing frameworks to design, implement, and optimize scalable data pipelines.Responsibilities : - Design, develop,...

  • Senior DevOps Engineer

    1 month ago

    Bangarapet, India Skan Full time

    Responsibilities & Deliverables: Guide the design and development of reusable Infrastructure as Code (IaC) components; create IaC test plans and execute testing across the lifecycle Deploy and support Cloud resources IaaS, PaaS, and/or SaaS, using infrastructure as code via CICD pipeline, K8 using Helm. Write automation scripts for building, deploying...

  • Data Enginer

    2 months ago

    Bengaluru, Karnataka, India NatWest Group Full time

    Our people work differently depending on their jobs and needs. From hybrid working to flexible hours, we have plenty of options that help our people to thrive. This role is based in India and as such all normal working days must be carried out in India. Join us as a Data Engineer - This is an exciting opportunity to use your technical expertise to...

  • Data Engineer

    4 weeks ago

    Karnataka, India Synechron Technologies Pvt. Ltd. Full time

    EducationDegree, Postgraduate in Computer Science or related field (or equivalent industry experience)ExperienceMinimum 5+ years of development and design experience in Java/Scala/Python with Flink, Beam (or Spark Streaming using Real Time data and not batch data) and Kafka.Technical Skills• Containerization and orchestration (Docker and...

  • Data Engineer

    4 weeks ago

    karnataka, India Synechron Technologies Pvt. Ltd. Full time

    Education Degree, Postgraduate in Computer Science or related field (or equivalent industry experience) Experience Minimum 5+ years of development and design experience in Java/Scala/Python with Flink, Beam (or Spark Streaming using Real Time data and not batch data) and Kafka. Technical Skills • Containerization and orchestration (Docker and...

  • Data Engineer

    4 weeks ago

    Karnataka, India Synechron Technologies Pvt. Ltd. Full time

    EducationDegree, Postgraduate in Computer Science or related field (or equivalent industry experience)ExperienceMinimum 5+ years of development and design experience in Java/Scala/Python with Flink, Beam (or Spark Streaming using Real Time data and not batch data) and Kafka.Technical Skills• Containerization and orchestration (Docker and...

  • Data Engineer

    3 weeks ago

    Karnataka, India Synechron Technologies Pvt. Ltd. Full time

    Education Degree, Postgraduate in Computer Science or related field (or equivalent industry experience) Experience Minimum 5+ years of development and design experience in Java/Scala/Python with Flink, Beam (or Spark Streaming using Real Time data and not batch data) and Kafka. Technical Skills • Containerization and orchestration (Docker and...

  • Engineer, Data

    4 weeks ago

    Karnataka, 560064, Bangalore, India Giant Eagle GCC Full time

    Job SummaryAs a Data Engineer on the Customer Loyalty & Experience team, you will be working on a team to bring customer-centric personalization to life. In this role, you will be empowered to develop data solutions in support of marketing analytics and data science by engaging with project managers, technology teams, analysts, and business partners to...

  • Engineer, Data

    4 weeks ago

    , Karnataka, India Giant Eagle GCC Full time

    Job SummaryAs a Data Engineer on the Customer Loyalty & Experience team, you will be working on a team to bring customer-centric personalization to life. In this role, you will be empowered to develop data solutions in support of marketing analytics and data science by engaging with project managers, technology teams, analysts, and business partners to...

  • Azure Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India DigitalCube Consultancy Full time

    **Azure Data Engineer** **Job Type - Full time** **Location - Bangalore/ Mumbai, India** **Responsibilities**: - Design, develop, and maintain scalable data pipelines using Azure Data Factory, Azure Databricks, and other Azure data services. - Utilize Databricks for big data analytics and real-time data processing, ensuring optimal performance and...

  • Infor Mfg, Qty, Engg Testing

    4 days ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Shift Timings: 1:00 PM IST to 11:00 PM IST Job Responsibilities: - Understand customizations to standard related to modules Mfg, Quality and Engineering - Understand integrations of other modules with Mfg, Quality and Engineering - Understand MES business process and integration with ERP LN - Write Functional, Regression and End to End Test Cases - Prepare...

  • Engineer Sr., Data

    1 month ago

    Karnataka, 560064, Bangalore, India Giant Eagle GCC Full time

    Job SummaryYou will be working on a team to bring customer-centric personalization to life. In this role, you will be empowered to develop data solutions in support of customer facing applications as well as analytics for internal teams and business. This leading technical and architecture role will collaborate with product managers, architects, technology...

  • Engineer Sr., Data

    1 month ago

    , Karnataka, India Giant Eagle GCC Full time

    Job SummaryYou will be working on a team to bring customer-centric personalization to life. In this role, you will be empowered to develop data solutions in support of customer facing applications as well as analytics for internal teams and business. This leading technical and architecture role will collaborate with product managers, architects, technology...

  • Engineer Sr., Data

    3 weeks ago

    , Karnataka, India Giant Eagle GCC Full time

    Job SummaryYou will be working on a team to bring customer-centric personalization to life. In this role, you will be empowered to develop data solutions in support of customer facing applications as well as analytics for internal teams and business. This leading technical and architecture role will collaborate with product managers, architects, technology...

  • Engineer Sr., Data

    4 weeks ago

    Karnataka, 560064, Bangalore, India Giant Eagle GCC Full time

    Job SummaryYou will be working on a team to bring customer-centric personalization to life. In this role, you will be empowered to develop data solutions in support of customer facing applications as well as analytics for internal teams and business. This leading technical and architecture role will collaborate with product managers, architects, technology...

  • Sdk Resource

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time

    **Req ID**: 280859 We are currently seeking a SDK Resource to join our team in BENGALURU, Karnātaka (IN-KA), India (IN). **Roles & Responsibilities**: Responsible for programming and software development using various programming languages and related tools and frameworks, reviewing code written by other programmers, requirement gathering, bug fixing,...