Big Data Engineer

1 month ago


Pune, Maharashtra, India Www.Huquo.com Full time

Big Data Engineer( Lead assistant manager-Manager)


Job Description :

Must Have :

Tech savvy engineer - willing and able to learn new skills, track industry trend.

- 5 - 7 years of total experience of solid data engineering experience, especially in Open Source, data-intensive, distributed environments with minimum 5 years of experience in big data related technologies like Spark, Hive, HBase, Hadoop, etc.

- Programming background - Mandatory- Scala, Spark and Java / Python

- Experience in following technologies - MapReduce, HDFS, YARN, Spark Streaming.

- Experience in creating e2e pipelines in Hadoop.

- Experience in SQL, Hive and NoSQL databases

- Experience in Linux / Unix platform

- Experience of cloud design patterns

- Experience in Jenkins, GitHub, Debugging the code, and error handling.

- Experience in Building end to end ETL Pipelines

- Self-starter & resourceful personality with the ability to manage pressure situations.

- Experience working with geographically distributed teams.

- Work on high- level business and technical problems, contribute to architectural decisions.

- Create the core framework/ templates that gives a starting point for the team members to get started with implementation/ execution.

Role & Responsibilities :

- Build Data and ETL pipelines in Hadoop.

- Support migration of data to the cloud using Big Data Technologies like Spark, Hive, Hadoop, Scala, Python

- Interact with customers daily to ensure smooth engagement.

- Exposure to GCP

- Responsible for timely and quality deliveries.

- Exposure to Scrum and Agile Development best practices

- Fulfil organization responsibilities

- Sharing knowledge and experience within the other groups in the organization, conducting various technical sessions and training.

(ref:hirist.tech)

  • Pune, Maharashtra, India sistemasgl Full time

    We are a digitally native company that helps organizations reinvent themselves and unleash their potential. We are the place where innovation, design and engineering meet scale. Globantis 20 years old, NYSE listed public organization with more than 27,000 employees worldwide working out of 25 countries globally. Positions Open: Azure Data EngineerExperience:...


  • Pune, Maharashtra, India Workday Full time

    Your work days are brighter here.At Workday, it all began with a conversation over breakfast. When our founders met at a sunny California diner, they came up with an idea to revolutionize the enterprise software market. And when we began to rise, one thing that really set us apart was our culture. A culture which was driven by our value of putting our people...

  • Senior Data Engineer

    3 weeks ago


    Pune, Maharashtra, India timesjobs Full time

    Senior Data EngineerPosition: Senior Data EngineerExperience: 8 yearsJob Location: PuneSkills Required:Developing, maintaining, and documenting reliable and scalable data infrastructure and data products in complex environmentsDevelopment experience in one or more object-oriented programming languages (e.g., Py-spark, Python, Java, C )Experience of working...


  • Pune, Maharashtra, India TeacherOn Full time

    I want to learn from someone who can train me well in Azure Data Engineering topics so that i can take a job as a Data Engineer and be confident while doing it in the real time. I have 6 years of experience in market Reasearch company and know the basics of SQL, Power Bi and Alteryx. If you think you can help me in my career transition, i would be happy to...


  • Pune, Maharashtra, India Ernst and Young Services Pvt. Ltd Full time

    To be successful in this role, you should meet the following requirements: (Must have Requirements)- Scala development and design using Scala 2.10+ or Java development and design using Java 1.8+.- Experience with most of the following technologies (Apache Hadoop, Scala, Apache Spark, Spark streaming, YARN, Kafka, Hive, Python, ETL frameworks, Map Reduce,...


  • Pune, Maharashtra, India ZF Group Full time

    Req ID 66501 Pune, India Main accountabilities, responsibilities and deliverables•While the data scientists spend time understanding the business, prototyping algorithms, concerning about the statistical aspects of the model, ML Engineer take the prototype requirements and optimize the implementation.•Ensure that ML models are implemented in the most...

  • Senior Data Engineer

    1 month ago


    Pune, Maharashtra, India Procore Full time

    Job DescriptionWe're looking for a Data Engineer to join Procore's data platform and who wants to make an impact. This is a great opportunity to work with an amazing team and help them achieve their mission to deliver actionable insights from data to the business. As a member of our Data team, you'll help define the future of our next-generation global data...


  • Pune, Maharashtra, India LTIMindtree Full time

    Job Description Testing consultant M6 8 10 years proven experience as a Solution Manager Test Manager Solution Architect on medium to largescale testing accounts across domains technologies Experience as a Testing SME leading the development of solutions for testing complex systems ensuring consistency with specified requirements agreed with both external...


  • Pune, Maharashtra, India FIS Global Full time

    Position Type : Full time Type Of Hire : Experienced (relevant combo of work and education) Education Desired : Bachelor of Engineering Travel Percentage : 0%About the team: We are a part of WorldPay, who invented payment facilitation, enabling technology companies to provide integrated payment solutions, significantly expanding access to modern payment...


  • Pune, Maharashtra, India timesjobs Full time

    What's the role?Hilti is investing massively into growing its software business and digitizing its customers processes. The Hilti IoT Data Platform provides key capabilities encompassing the collection, processing, and analysis of tool data to enable business use cases supporting our growth ambition. As a Data Engineer within the IoT & Analytics team, your...

  • Python Developer

    1 month ago


    Pune, Maharashtra, India Corpxcel Consulting Full time

    Location : Pune OnlyRole type : IC and completely hands on Qualifications :- 2+ years of Java/Scala development experience.- 2+ years of SQL and NoSQL experience (handling structured and unstructured data).- 1+ years of extensive experience with Spark Processing engine.- 1+ years of experience with Big data tools / technologies/ Streaming (Hive, Impala,...


  • Pune, Maharashtra, India NTT Data Vertex Software Inc. Full time

    Apply for Digital Engineering Staff Engineer, Career Progress Consultants in Pune for Year of Experience on


  • Pune, Maharashtra, India Deloitte India Full time

    Job Description : - Software design, Scala & Spark, PySpark & Python development, automated testing of new and existing components in an Agile, DevOps and dynamic environment Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting.- Implement the tools and processes, handling performance, scale,...


  • Pune, Maharashtra, India Hevo Data Full time

    About HevoHevo ) is a simple, intuitive, and powerful No-code Data Pipeline platform that enables companies to consolidate data from multiple software for faster analytics.Hevo powers data analytics for 2000+ data-driven companies across multiple industry verticals including Shopify, DoorDash, Deliverr, Neo4J, Arhaus, Marqeta, and Postman. By automating...

  • Data Engineer

    1 week ago


    Pune, Maharashtra, India Programming.com Full time

    Skill Set :Languages : Java, Scala, PythonTechnology : Azure Cloud, Databricks , Spark-Scala, PySpark, Kafka, SQL, Linux/UnixNice to have : Hadoop, Hive, Power BINote : Databricks - Knowledge of Databricks CLI , REST APIs, Spark config , Performance and Tuning.These are the must skill to work/enhance and support existing tools and frameworks and to do day to...

  • Data Engineer

    3 weeks ago


    Pune, Maharashtra, India timesjobs Full time

    Skills Required:Experience developing, maintaining, and documenting reliable and scalable data infrastructure and data products in complex environmentsDevelopment experience in one or more object-oriented programming languages (e.g., Py-spark, Python, Java, C )Experience working in a cloud environment AWS / AzureExcellent data modeling and dimensional...


  • Pune, Maharashtra, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Industry Consulting Sr. Manager to join our team in Pune, Mahārāshtra (IN-MH), India (IN).People & Organization...


  • Pune, Maharashtra, India TeacherOn Full time

    I want to learn about basics of data engineering workflow including ,Kafka, Hadoop, pyspark , from non technical background but need it as I have switched to Business Intelligence role.Level: BeginnerGender Preference: NoneMeeting options:Available online - via skype etc.


  • Pune, Maharashtra, India NTT DATA Services Full time

    Developing test automation code and scripts to validate feature code qualityContributing to the development and maintenance of best practices regarding test data, test plans, test repositories, and test framework infrastructureDeveloping mechanisms for collecting, interpreting, tracking, and reporting on test execution and quality metricsCollaborating with...


  • Pune, Maharashtra, India MMC Corporate Full time

    Corporate Information SolutionsThe Data Visualization Specialist will take a user-centric lens to make complex data accessible at the essential decision points for business leaders and domain experts through interactive data visualizations and collaborative data analysis systems. The data visualization products developed should make the data accessible,...