Data Engineer

4 weeks ago

Bangarapet, India RiskInsight Consulting Pvt Ltd Full time

About the Role :

We are seeking a skilled and experienced Big Data Engineer to join our team and play a crucial role in building and maintaining our big data infrastructure.

You will leverage your expertise in the Hadoop ecosystem and data processing frameworks to design, implement, and optimize scalable data pipelines.

Responsibilities :

- Design, develop, and maintain efficient data pipelines for large-scale data processing.

- Utilize your proficiency in HDFS, HiveQL, and PySpark to create and optimize data processing workflows.

- Perform data wrangling and transformation tasks to ensure data quality and consistency.

- Write efficient SQL queries for data manipulation and retrieval.

- Automate tasks and schedule jobs using Linux shell scripting.

- Develop and maintain Python scripts for data manipulation, analysis, and automation tasks.

- Collaborate with data analysts and data scientists to understand data needs and translate them into technical solutions.

- Troubleshoot and debug big data processing issues to ensure smooth system operation.

- Continuously monitor and optimize data pipelines for performance and scalability.

- Advocate for best practices in data management and big data technologies.

- Stay up-to-date on the latest trends and innovations in the big data and analytics landscape.

Qualifications :

- 4-8 years of experience as a Data Engineer or a similar role.

- Strong expertise in Hadoop ecosystem components, including HDFS, HiveQL, and YARN.

- Proven experience in developing and optimizing data processing workflows using PySpark.

- Proficiency in SQL for data manipulation and querying.

- Hands-on experience with Linux shell scripting for automation and job scheduling.

- Solid understanding of Python programming language and libraries for data manipulation and analysis.

- Experience with version control systems (Git) is a plus.

- Knowledge of CI/CD pipelines is a plus.

- Excellent communication and collaboration skills to work effectively with cross-functional teams.

- Strong problem-solving skills and a keen eye for detail.

- Proactive and eager to learn new technologies and stay updated in the big data field.

(ref:hirist.tech)

  • Networking Advisor

    3 weeks ago

    Bangarapet, India NTT DATA Full time

    **Req ID**:223622 We are currently seeking a Networking Advisor to join our team in Delhi, Uttar Pradesh (IN-UP), India (IN). **Position overview** The position requires, day to day operational support for requests and incidents on all network related aspects involving multiple technologies such as: wireless, switching, routing, sdwan, and high level...

  • Senior DevOps Engineer

    1 month ago

    Bangarapet, India Skan Full time

    Responsibilities & Deliverables: Guide the design and development of reusable Infrastructure as Code (IaC) components; create IaC test plans and execute testing across the lifecycle Deploy and support Cloud resources IaaS, PaaS, and/or SaaS, using infrastructure as code via CICD pipeline, K8 using Helm. Write automation scripts for building, deploying...

  • Data Engineer

    2 months ago

    Karnataka/Bangarapet, IN RiskInsight Consulting Pvt Ltd Full time

    About the Role :We are seeking a skilled and experienced Big Data Engineer to join our team and play a crucial role in building and maintaining our big data infrastructure. You will leverage your expertise in the Hadoop ecosystem and data processing frameworks to design, implement, and optimize scalable data pipelines.Responsibilities : - Design, develop,...

  • Data Engineer

    2 months ago

    Karnataka/Bangarapet, India RiskInsight Consulting Pvt Ltd Full time

    About the Role :We are seeking a skilled and experienced Big Data Engineer to join our team and play a crucial role in building and maintaining our big data infrastructure. You will leverage your expertise in the Hadoop ecosystem and data processing frameworks to design, implement, and optimize scalable data pipelines.Responsibilities : - Design, develop,...

  • Data Engineer

    4 weeks ago

    Karnataka/Bangarapet, India RiskInsight Consulting Pvt Ltd Full time

    About the Role :We are seeking a skilled and experienced Big Data Engineer to join our team and play a crucial role in building and maintaining our big data infrastructure. You will leverage your expertise in the Hadoop ecosystem and data processing frameworks to design, implement, and optimize scalable data pipelines.Responsibilities : - Design, develop,...

  • Data Engineer

    4 weeks ago

    Karnataka/Bangarapet, IN RiskInsight Consulting Pvt Ltd Full time

    About the Role :We are seeking a skilled and experienced Big Data Engineer to join our team and play a crucial role in building and maintaining our big data infrastructure. You will leverage your expertise in the Hadoop ecosystem and data processing frameworks to design, implement, and optimize scalable data pipelines.Responsibilities : - Design, develop,...