Senior Data Engineer

2 weeks ago


PuneMysore, IN IT Full time

We are seeking a talented Senior Data Engineer with a strong background in developing ETLs and data pipelines using Apache Spark with the PySpark library.

The ideal candidate will have proven experience in data modeling for data warehousing use cases, designing patterns, and building highly scalable and secure solutions.

Additionally, experience with Azure Stack, workflow management tools like Apache Airflow, Scrum, and DevOps methodology is required.

Responsibilities :

- Develop and maintain ETL processes and data pipelines using Apache Spark with PySpark library.

- Design and implement data models for data warehousing use cases, ensuring scalability and security.

- Collaborate with cross-functional teams to gather and understand data requirements.

- Implement best practices for data processing, storage, and retrieval.

- Utilize Azure Stack for cloud-based data solutions.

- Develop and maintain workflow automation using Apache Airflow.

- Work closely with stakeholders to understand business requirements and translate them into technical solutions.

- Follow Scrum and DevOps methodologies to deliver high-quality solutions in a timely manner.

- Continuously learn and stay updated on emerging technologies and industry trends.

- Foster effective teamwork and collaboration within the engineering team.

Requirements :

- Bachelor's/Master's degree in Computer Science, Engineering, or a related field.

- Proven experience as a Data Engineer with a focus on developing ETLs and data pipelines using Apache Spark with PySpark library.

- Strong understanding of data modeling concepts and experience designing data warehouses.

- Experience with Azure Stack for cloud-based data solutions.

- Proficiency in workflow management tools like Apache Airflow.

- Familiarity with Scrum and DevOps methodologies.

- Open-minded, flexible, self-reflective, and sincere approach to work.

- Eager to learn and grow in a dynamic environment.

- Excellent communication and collaboration skills.

(ref:hirist.tech)

  • Bangalore/Pune, IN Recrud Full time

    Senior Big Data Engineer Position. Experience Required: 6 -10 Years. Employment Type: Full-Time. About Our Client:. Our client is a distinguished product engineering consulting firm based in Bangalore, dedicated to providing innovative offshore development solutions. With over half a century of combined expertise in product engineering and consulting...

  • AWS Data Engineer

    3 weeks ago


    Bangalore/Pune, IN Technomech Placements Full time

    This is a full-time hybrid role for a Senior AWS Data Engineer. The Senior AWS Data Engineer will be responsible for day-to-day tasks such as data engineering, data modeling, ETL (Extract Transform Load), data warehousing, and data analytics.Qualifications :- Data Engineering, Data Modeling, and ETL (Extract Transform Load) skills- Data Warehousing and Data...


  • Bangalore/Mysore/Coimbatore/Any Location, IN Aezion.Inc Full time

    Job Title : Enterprise Data Architect-with Healthcare experienceLocation : :Job Overview :We are looking for a senior Data Architect /data modeler to analyze Healthcare business and data requirements, and model data structures to enable the technical solution and implementation within a Healthcare Provider organization. This position will involve working as...

  • Senior Data Engineer

    5 hours ago


    Bangalore/Hyderabad/Pune/Gurgaon/Gurugram/Noida/Greater Noida, IN Publicis Sapient Full time

    Publicis Sapient Overview : We at Publicis Sapient, enable our clients to thrive in Next and to create business value through expert strategies, customer-centric experience design, and world-class product engineering.The future of business is disruptive, transformative and becoming digital to the core.In our 20 + years in IT, never before have we seen such a...

  • Senior Data Engineer

    1 month ago


    Bangalore/Kolkata/Pune, IN Nilasu consulting Full time

    Summary :- We are seeking a highly skilled and experienced Senior Data Engineer to join our team.- You will play a key role in designing, developing, and maintaining large-scale data processing pipelines using Python and Spark/PySpark.- Your expertise in distributed computing frameworks and DevOps tools will be instrumental in building efficient and scalable...

  • Senior Data Engineer

    1 month ago


    Pune/Chandigarh/Mohali, IN Tekfortune IT India Pvt. Ltd. Full time

    Role/Title : Senior Data EngineerExperience : 6-10 Years' experienceLocation : PuneNotice period : 0-30days onlyKey Skills : SSIS, SQL, Azure SynapseJob Summary :As a Senior Data Engineer, you will support the European Market using the Windows & Azure suite of Analytics & Data platforms. The focus of the role is on the technical aspects and...


  • Pune/Hyderabad/Gurgaon/Gurugram, IN Huquo Full time

    About Clairvoyant :Clairvoyant is a global technology consulting and services company founded in 2012, headquartered in Chandler, US, and has delivery centers across the globe. We help organizations maximize the value of data by providing data engineering, analytics, machine learning, and user experience consulting and development projects to multiple...

  • Senior Data Engineer

    1 month ago


    Hyderabad/Pune/Bangalore, IN iMind Your Business Solutions Private Limited Full time

    Job Description :We are hiring Data Engineers for the leaders In Analytics & Cloud Applications, headquartered in San Francisco and have offices in India (Bangalore, Hyderabad and Pune).Our client provides strategic guidance and technology systems for clients wishing to solve their most complex and interesting business challenges involving cloud...


  • Pune, MH, IN BP Energy Full time

    Entity: Innovation & EngineeringJob Family Group: IT&S GroupJob Description: Role Synopsis:bp is reinventing itself and digital capability is at the core of this vision. As a Senior Enterprise Technology Engineer you are a digital expert bringing deep specialist expertise to bp. Enterprise Technology Engineers work on the strategic technology platforms we...


  • Pune/Bangalore, IN Hiret Full time

    Job Description :- Develop high performance cloud native Java microservices.- Mentor/guide junior developers in completing their stories.- Working closely with Business Analysts, end users and architects to build flexible enterprise software.Role : Senior Software engineer - JavaJob location : Pune & Bangalore- We are reimagining products and redesigning...

  • Senior Data Engineer

    1 month ago


    Chennai/Bangalore/Hyderabad/Pune, IN Resourcetree Full time

    Job Description :Data engineering role requires creating and managing technological infrastructure of a data platform, be in- charge / involved in architecting, building, and managing data flows / pipelines and construct data storages (noSQL, SQL), tools to work with big data (Hadoop, Kafka), and integration tools to connect sources or other databases.Role...

  • Data Engineer

    2 weeks ago


    Nagpur/Pune, IN SPN Globe Full time

    Job Description :Mandatory Skills : Azure Data Engineer, ADF, Synapse, Databricks, Azure DevOps, Data lake, SQL, and Responsibilities :Designing and implementing data storage solutions on AzureBuilding and maintaining data pipelines for data integration and processingEnsuring data quality and accuracy through testing and validationOptimizing data processing...


  • Pune, MH, IN BP Energy Full time

    Entity: FinanceJob Family Group: Finance GroupJob Description: KEY ACCOUNTABILITIES :Data quality framework: we demonstrate deep understanding of data value chain, encompassing Critical Data Element concepts, maintain data quality, DQ critical metrics/Measures and DQ remediation. Perform Data concerns assessments to aid improvements in operational process...

  • Senior Data Scientist

    3 weeks ago


    Bangalore/Hyderabad/Chennai/Pune, IN Nilasu consulting Full time

    About the Role :We are searching for a talented and experienced Senior Data Scientist to join our team and play a key role in leveraging the power of machine learning (ML) and artificial intelligence (AI) to extract valuable insights from data. In this role, you will be responsible for the entire data science lifecycle, from data acquisition and wrangling to...


  • Bangalore/Mumbai/Pune, IN Techno Wise Full time

    Job Description:We are seeking a highly skilled Senior Azure Application Engineer to join our dynamic team. The ideal candidate should have extensive experience working with various Azure services and possess a strong background in application development. As a Senior Azure Application Engineer, you will be responsible for designing, developing, and...

  • GBS Data Operations

    2 days ago


    Pune, MH, IN BP Energy Full time

    Entity: FinanceJob Family Group: Finance GroupJob Description: Purpose of Role:Assess the quality of engineering data, documents and models in ALIM proposing corrections to engineering to improve integrity of the design associated to assets and projects across P&O globally.Implement effective controls for internal delegation, audit and control of the quality...

  • Data Engineer

    4 weeks ago


    Pune/Hyderabad, IN Promaynaov Advisory Services Pvt Ltd Full time

    Role : Data EngineerLocation : Pune/HyderabadIn this role, you will : Principal Responsibilities :As a key member of the technical team alongside Engineers, Data Scientists and Data Users, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process - Software design, Scala & Spark development,...

  • AWS Data Engineer

    1 month ago


    Bangalore/Pune/Chennai, IN Change leaders Full time

    Job Description : AWS Glue + Python PysparkJob Overview :- Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform.- This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...

  • Data Engineer

    2 weeks ago


    Hyderabad/Chennai/Pune, IN Freelancer HR Full time

    Description :Job Description :- Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure Data bricks and/or HDInsights- Experience with other Open Source big data products Hadoop (incl. Hive, Pig, Impala)- Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB,...


  • Gurgaon/Gurugram/Pune/Hyderabad, IN Huquo Full time

    Job Description : Responsibilities : - Hands-on experience with any ETL tool like Informatica, Talend, SSIS, Data Lake, Azure Data Factory etc. OR Strong hands-on experience on Azure, AWS, GCP- Write advance and complex SQL queries with solid experience in database management- Ability to work effectively as member of a team, with minimal guidance- Understand...