Data Engineer

4 weeks ago

PuneHyderabad, IN Promaynaov Advisory Services Pvt Ltd Full time

Role : Data Engineer

Location : Pune/Hyderabad

In this role, you will :

Principal Responsibilities :

As a key member of the technical team alongside Engineers, Data Scientists and Data Users, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process

- Software design, Scala & Spark development, automated testing of new and existing components in an Agile, DevOps and dynamic environment

- Promoting development standards, code reviews, mentoring, knowledge sharing

- Production support & troubleshooting.

- Implement the tools and processes, handling performance, scale, availability, accuracy and monitoring

- Liaison with BAs to ensure that requirements are correctly interpreted and implemented.

- Participation in regular planning and status meetings. Input to the development process - through the involvement in Sprint reviews and retrospectives. Input into system architecture and design.

- Peer code reviews.

- To be successful in this role, you should meet the following requirements: (Must have Requirements)

- Scala development and design using Scala 2.10+ or Java development and design using Java 1.8+.

- Experience with most of the following technologies (Apache Hadoop, Scala, Apache Spark, Spark streaming, YARN, Kafka, Hive, Python, ETL frameworks, Map Reduce, SQL, RESTful services).

- Sound knowledge on working Unix/Linux Platform

- Hands-on experience building data pipelines using Hadoop components - Hive, Spark, Spark SQL.

- Experience with industry standard version control tools (Git, GitHub), automated deployment tools (Ansible & Jenkins) and requirement management in JIRA.

- Understanding of big data modelling techniques using relational and non-relational techniques

- Experience on Debugging the Code issues and then publishing the highlighted differences to the development team/Architects;

- The successful candidate will also meet the following requirements


Good to have Requirements :

- Experience with time-series/analytics dB's such as Elastic search.

- Experience with scheduling tools such as Airflow, Control-M.

- Understanding or experience of Cloud design patterns

- Exposure to DevOps & Agile Project methodology such as Scrum and Kanban.

- Experience with developing Hive QL, UDF's for analysing semi structured/structured datasets.


Immediate joiners preferred, max 45 days (must be serving notice)

(ref:hirist.tech)

  • AWS Data Engineer

    1 month ago

    Pune/Hyderabad, IN Huquo Full time

    Role : Data EngineerMust-Have : - 4 to 6 years of Relevant experience in Data Engineering and delivery.- 3 to 4 years of relevant work experience in large-scale Data applications- AWS Glue is mandatory.- Strong 4-6 years of working experience in SQL Spark Python, Snowflake, and AWS cloud knowledge.- Good aptitude, strong problem-solving abilities, and...

  • Senior Data Engineer

    1 month ago

    Chennai/Bangalore/Hyderabad/Pune, IN Resourcetree Full time

    Job Description :Data engineering role requires creating and managing technological infrastructure of a data platform, be in- charge / involved in architecting, building, and managing data flows / pipelines and construct data storages (noSQL, SQL), tools to work with big data (Hadoop, Kafka), and integration tools to connect sources or other databases.Role...

  • AWS Data Engineer

    1 week ago

    Bangalore/Pune/Greater Noida, IN HyrEzy Talent Solutions Full time

    Job Title : AWS Data EngineerLocation : Pune , Bangalore, Greater Noida, TrichyResponsibilities :Design, build, and maintain scalable data architectures on AWS cloud platform.Develop and implement data pipelines for ingesting, processing, and transforming large volumes of data from diverse sources.Design and optimize data models for efficient storage,...

  • Senior Cloud Data Engineer

    1 month ago

    Hyderabad/Bangalore/Chennai/Vishakhapatnam/Vizag, IN Mouri Tech Ltd Full time

    Senior Cloud Data Engineer (AWS, Python, Pyspark) Job Title : Senior AWS Cloud Data EngineerWork Location : Hyderabad/Kolhapur/Bangalore/Chennai Experience : 6 to 9 yearsRequired Skills : Python, Pyspark, AWS services such as DMS, SAM, Glue, Redshift, Lamda S3 with Apache Airflow, CI/CD, :- Minimum of 6-9 Experience with the core AWS Services and setting up...

  • Pipeline Engineer

    2 days ago

    Pune, MH, IN BP Energy Full time

    Entity: Production & OperationsJob Family Group: Engineering GroupJob Summary: bp Technical Solutions India (TSI) center in Pune, aims to build on bp’s existing engineering and technical strengths to deliver high quality services to its hydrocarbons and new energy businesses worldwide! TSI brings together diverse engineering capability to provide technical...

  • AWS Cloud Architect

    1 month ago

    Bangalore/Gurgaon/Gurugram/Pune, IN HuQuo Consulting Pvt. Ltd. Full time

    Job Responsibilities : Data Architecture and Cloud Strategy :Develop and maintain a comprehensive data architecture and cloud strategy that aligns with the organization's goals and needs.Design, implement and manage cloud-based data infrastructure on AWS, ensuring scalability, reliability, and cost-efficiency.Utilize AWS services (S3, Glue, EMR,...

  • Reservoir Engineer

    2 days ago

    Pune, MH, IN BP Energy Full time

    Entity: Production & OperationsJob Family Group: Subsurface GroupJob Summary: BP Technical Solutions India (TSI) center in Pune, aims to build on bp’s existing engineering and technical strengths to deliver high quality services to its hydrocarbons and new energy businesses worldwide. TSI brings together diverse engineering capability to provide technical...

  • Subsea Controls Engineer

    2 days ago

    Pune, MH, IN BP Energy Full time

    Entity: Production & OperationsJob Family Group: Engineering GroupJob Summary: bp Technical Solutions India (TSI) center in Pune, aims to build on bp’s existing engineering and technical strengths to deliver high quality services to its hydrocarbons and new energy businesses worldwide! TSI brings together diverse engineering capability to provide technical...

  • Reservoir Engineer

    2 days ago

    Pune, MH, IN BP Energy Full time

    Entity: Production & OperationsJob Family Group: Subsurface GroupJob Description: We are a global energy business involved in every aspect of the energy system. We are working towards delivering light, heat, and mobility to millions of people every day. We are one of the very few companies equipped to solve some of the big challenges that matter for the...

  • DevOps Engineer

    1 month ago

    Bangalore/Pune/Hyderabad, IN Shework Full time

    Position Overview :As a DevOps Engineer specializing in Virtualization Infrastructure, you will play a critical role in designing, implementing, and managing our virtualization infrastructure environment. You will leverage your expertise in VMware and VDI technologies to ensure the stability, performance, and scalability of our virtualized systems....

  • Sr. Subsea Riser Engineer

    2 days ago

    Pune, MH, IN BP Energy Full time

    Entity: Production & OperationsJob Family Group: Engineering GroupJob Summary: bp Technical Solutions India (TSI) center in Pune, aims to build on bp’s existing engineering and technical strengths to deliver high quality services to its hydrocarbons and new energy businesses worldwide! TSI brings together diverse engineering capability to provide technical...

  • Subsea Riser Engineer

    1 day ago

    Pune, MH, IN BP Energy Full time

    Entity: Production & OperationsJob Family Group: Engineering GroupJob Description: About bpOur purpose is to bring together people, energy and markets to power and navigate a changing world.In a time of constant change and possibility we need new talent to pursue commercial opportunities, fueled by special insight and expertise. We’re always aspiring for...

  • Flow Assurance Engineer

    2 days ago

    Pune, MH, IN BP Energy Full time

    Entity: Production & OperationsJob Family Group: Engineering GroupJob Summary: bp Technical Solutions India (TSI) center in Pune, aims to build on bp’s existing engineering and technical strengths to deliver high quality services to its hydrocarbons and new energy businesses worldwide! TSI brings together diverse engineering capability to provide technical...

  • Sr. Subsea Hardware Engineer

    2 days ago

    Pune, MH, IN BP Energy Full time

    Entity: Production & OperationsJob Family Group: Engineering GroupJob Summary: bp Technical Solutions India (TSI) center in Pune, aims to build on bp’s existing engineering and technical strengths to deliver high quality services to its hydrocarbons and new energy businesses worldwide! TSI brings together diverse engineering capability to provide technical...

  • Rotating Equipment Engineer

    2 days ago

    Pune, MH, IN BP Energy Full time

    Entity: Production & OperationsJob Family Group: Engineering GroupJob Description: What you will deliverProvide Rotating Equipment Engineering support to reduce risk, deliver operations support by working closely with the Site and the other squadsProvides rotating equipment engineering (REE) expertise to other teams by performing and reviewing complex, or...

  • Mechanical Engineer

    1 day ago

    Pune, MH, IN BP Energy Full time

    Entity: Production & OperationsJob Family Group: Engineering GroupJob Description: What you will deliverProvide Mechanical Engineering support to reduce risk, deliver operations support by working closely with the Region Production Delivery and Support Unit Squads and SitesProvides ME expertise to MoCs, brownfield projects and modifications, TAR and outages,...

  • Materials & Corrosion Engineer - TSI

    2 days ago

    Pune, MH, IN BP Energy Full time

    Entity: Production & OperationsJob Family Group: Engineering GroupJob Summary: bp Technical Solutions India (TSI) centre in Pune, aims to build on bp’s existing engineering and technical strengths to deliver high quality services to its hydrocarbons and new energy businesses worldwide. TSI brings together diverse engineering capability to provide technical...

  • PSI Engineer

    2 days ago

    Pune, MH, IN BP Energy Full time

    Entity: Production & OperationsJob Family Group: Engineering GroupJob Description: bp Technical Solutions India (TSI) centre in Pune, aims to build on bp’s existing engineering and technical strengths to deliver high quality services to its hydrocarbons and new energy businesses worldwide. TSI brings together diverse engineering capability to provide...

  • Sr. Rotating Equipment Engineer

    2 days ago

    Pune, MH, IN BP Energy Full time

    Entity: Production & OperationsJob Family Group: Engineering GroupJob Description: What you will deliverProvide Rotating Equipment Engineering support to reduce risk, deliver operations support by working closely with the Site and the other squadsProvides rotating equipment engineering (REE) expertise to other teams by performing and reviewing complex, or...

  • Materials & Corrosion Engineer - TSI

    2 days ago

    Pune, MH, IN BP Energy Full time

    Entity: Production & OperationsJob Family Group: Engineering GroupJob Summary: bp Technical Solutions India (TSI) centre in Pune, aims to build on bp’s existing engineering and technical strengths to deliver high quality services to its hydrocarbons and new energy businesses worldwide. TSI brings together diverse engineering capability to provide technical...