Senior Data Engineer
2 weeks ago
We are seeking a talented Senior Data Engineer with a strong background in developing ETLs and data pipelines using Apache Spark with the PySpark library.
The ideal candidate will have proven experience in data modeling for data warehousing use cases, designing patterns, and building highly scalable and secure solutions.
Additionally, experience with Azure Stack, workflow management tools like Apache Airflow, Scrum, and DevOps methodology is required.
Responsibilities :
- Develop and maintain ETL processes and data pipelines using Apache Spark with PySpark library.
- Design and implement data models for data warehousing use cases, ensuring scalability and security.
- Collaborate with cross-functional teams to gather and understand data requirements.
- Implement best practices for data processing, storage, and retrieval.
- Utilize Azure Stack for cloud-based data solutions.
- Develop and maintain workflow automation using Apache Airflow.
- Work closely with stakeholders to understand business requirements and translate them into technical solutions.
- Follow Scrum and DevOps methodologies to deliver high-quality solutions in a timely manner.
- Continuously learn and stay updated on emerging technologies and industry trends.
- Foster effective teamwork and collaboration within the engineering team.
Requirements :
- Bachelor's/Master's degree in Computer Science, Engineering, or a related field.
- Proven experience as a Data Engineer with a focus on developing ETLs and data pipelines using Apache Spark with PySpark library.
- Strong understanding of data modeling concepts and experience designing data warehouses.
- Experience with Azure Stack for cloud-based data solutions.
- Proficiency in workflow management tools like Apache Airflow.
- Familiarity with Scrum and DevOps methodologies.
- Open-minded, flexible, self-reflective, and sincere approach to work.
- Eager to learn and grow in a dynamic environment.
- Excellent communication and collaboration skills.
-
Senior Big Data Engineer
1 month ago
Bangalore/Pune, IN Recrud Full timeSenior Big Data Engineer Position. Experience Required: 6 -10 Years. Employment Type: Full-Time. About Our Client:. Our client is a distinguished product engineering consulting firm based in Bangalore, dedicated to providing innovative offshore development solutions. With over half a century of combined expertise in product engineering and consulting...
-
AWS Data Engineer
3 weeks ago
Bangalore/Pune, IN Technomech Placements Full timeThis is a full-time hybrid role for a Senior AWS Data Engineer. The Senior AWS Data Engineer will be responsible for day-to-day tasks such as data engineering, data modeling, ETL (Extract Transform Load), data warehousing, and data analytics.Qualifications :- Data Engineering, Data Modeling, and ETL (Extract Transform Load) skills- Data Warehousing and Data...
-
Aezion - Senior Data Architect - Data Modeling
1 month ago
Bangalore/Mysore/Coimbatore/Any Location, IN Aezion.Inc Full timeJob Title : Enterprise Data Architect-with Healthcare experienceLocation : :Job Overview :We are looking for a senior Data Architect /data modeler to analyze Healthcare business and data requirements, and model data structures to enable the technical solution and implementation within a Healthcare Provider organization. This position will involve working as...
-
Senior Data Engineer
5 hours ago
Bangalore/Hyderabad/Pune/Gurgaon/Gurugram/Noida/Greater Noida, IN Publicis Sapient Full timePublicis Sapient Overview : We at Publicis Sapient, enable our clients to thrive in Next and to create business value through expert strategies, customer-centric experience design, and world-class product engineering.The future of business is disruptive, transformative and becoming digital to the core.In our 20 + years in IT, never before have we seen such a...
-
Senior Data Engineer
1 month ago
Bangalore/Kolkata/Pune, IN Nilasu consulting Full timeSummary :- We are seeking a highly skilled and experienced Senior Data Engineer to join our team.- You will play a key role in designing, developing, and maintaining large-scale data processing pipelines using Python and Spark/PySpark.- Your expertise in distributed computing frameworks and DevOps tools will be instrumental in building efficient and scalable...
-
Senior Data Engineer
1 month ago
Pune/Chandigarh/Mohali, IN Tekfortune IT India Pvt. Ltd. Full timeRole/Title : Senior Data EngineerExperience : 6-10 Years' experienceLocation : PuneNotice period : 0-30days onlyKey Skills : SSIS, SQL, Azure SynapseJob Summary :As a Senior Data Engineer, you will support the European Market using the Windows & Azure suite of Analytics & Data platforms. The focus of the role is on the technical aspects and...
-
Lead/Senior Azure Data Engineer
3 weeks ago
Pune/Hyderabad/Gurgaon/Gurugram, IN Huquo Full timeAbout Clairvoyant :Clairvoyant is a global technology consulting and services company founded in 2012, headquartered in Chandler, US, and has delivery centers across the globe. We help organizations maximize the value of data by providing data engineering, analytics, machine learning, and user experience consulting and development projects to multiple...
-
Senior Data Engineer
1 month ago
Hyderabad/Pune/Bangalore, IN iMind Your Business Solutions Private Limited Full timeJob Description :We are hiring Data Engineers for the leaders In Analytics & Cloud Applications, headquartered in San Francisco and have offices in India (Bangalore, Hyderabad and Pune).Our client provides strategic guidance and technology systems for clients wishing to solve their most complex and interesting business challenges involving cloud...
-
Senior Enterprise Technology Engineer
2 days ago
Pune, MH, IN BP Energy Full timeEntity: Innovation & EngineeringJob Family Group: IT&S GroupJob Description: Role Synopsis:bp is reinventing itself and digital capability is at the core of this vision. As a Senior Enterprise Technology Engineer you are a digital expert bringing deep specialist expertise to bp. Enterprise Technology Engineers work on the strategic technology platforms we...
-
Senior Software Engineer
1 month ago
Pune/Bangalore, IN Hiret Full timeJob Description :- Develop high performance cloud native Java microservices.- Mentor/guide junior developers in completing their stories.- Working closely with Business Analysts, end users and architects to build flexible enterprise software.Role : Senior Software engineer - JavaJob location : Pune & Bangalore- We are reimagining products and redesigning...
-
Senior Data Engineer
1 month ago
Chennai/Bangalore/Hyderabad/Pune, IN Resourcetree Full timeJob Description :Data engineering role requires creating and managing technological infrastructure of a data platform, be in- charge / involved in architecting, building, and managing data flows / pipelines and construct data storages (noSQL, SQL), tools to work with big data (Hadoop, Kafka), and integration tools to connect sources or other databases.Role...
-
Data Engineer
2 weeks ago
Nagpur/Pune, IN SPN Globe Full timeJob Description :Mandatory Skills : Azure Data Engineer, ADF, Synapse, Databricks, Azure DevOps, Data lake, SQL, and Responsibilities :Designing and implementing data storage solutions on AzureBuilding and maintaining data pipelines for data integration and processingEnsuring data quality and accuracy through testing and validationOptimizing data processing...
-
Senior Data Quality Analyst
6 days ago
Pune, MH, IN BP Energy Full timeEntity: FinanceJob Family Group: Finance GroupJob Description: KEY ACCOUNTABILITIES :Data quality framework: we demonstrate deep understanding of data value chain, encompassing Critical Data Element concepts, maintain data quality, DQ critical metrics/Measures and DQ remediation. Perform Data concerns assessments to aid improvements in operational process...
-
Senior Data Scientist
3 weeks ago
Bangalore/Hyderabad/Chennai/Pune, IN Nilasu consulting Full timeAbout the Role :We are searching for a talented and experienced Senior Data Scientist to join our team and play a key role in leveraging the power of machine learning (ML) and artificial intelligence (AI) to extract valuable insights from data. In this role, you will be responsible for the entire data science lifecycle, from data acquisition and wrangling to...
-
Senior Azure Application Engineer
1 month ago
Bangalore/Mumbai/Pune, IN Techno Wise Full timeJob Description:We are seeking a highly skilled Senior Azure Application Engineer to join our dynamic team. The ideal candidate should have extensive experience working with various Azure services and possess a strong background in application development. As a Senior Azure Application Engineer, you will be responsible for designing, developing, and...
-
GBS Data Operations
2 days ago
Pune, MH, IN BP Energy Full timeEntity: FinanceJob Family Group: Finance GroupJob Description: Purpose of Role:Assess the quality of engineering data, documents and models in ALIM proposing corrections to engineering to improve integrity of the design associated to assets and projects across P&O globally.Implement effective controls for internal delegation, audit and control of the quality...
-
Data Engineer
4 weeks ago
Pune/Hyderabad, IN Promaynaov Advisory Services Pvt Ltd Full timeRole : Data EngineerLocation : Pune/HyderabadIn this role, you will : Principal Responsibilities :As a key member of the technical team alongside Engineers, Data Scientists and Data Users, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process - Software design, Scala & Spark development,...
-
AWS Data Engineer
1 month ago
Bangalore/Pune/Chennai, IN Change leaders Full timeJob Description : AWS Glue + Python PysparkJob Overview :- Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform.- This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...
-
Data Engineer
2 weeks ago
Hyderabad/Chennai/Pune, IN Freelancer HR Full timeDescription :Job Description :- Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure Data bricks and/or HDInsights- Experience with other Open Source big data products Hadoop (incl. Hive, Pig, Impala)- Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB,...
-
Manager/Senior Manager
1 month ago
Gurgaon/Gurugram/Pune/Hyderabad, IN Huquo Full timeJob Description : Responsibilities : - Hands-on experience with any ETL tool like Informatica, Talend, SSIS, Data Lake, Azure Data Factory etc. OR Strong hands-on experience on Azure, AWS, GCP- Write advance and complex SQL queries with solid experience in database management- Ability to work effectively as member of a team, with minimal guidance- Understand...