ETL Data Engineer
3 weeks ago
Job Description: ETL - Data Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.
- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness
- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.
- Diagnose system performance issue related to data processing and implement solution to address them.
- Collaborate with other teams to ensure successful integration of data pipelines into larger system architecture requirement.
- Maintain integrity and quality across all pipelines and environments.
- Understand and follow secure coding practice to make sure code is not vulnerable
Skills:
- Should have Strong background in Software development with experience in ingest, transform and store data from large datasets using Pyspark in Azure Databricks with strong knowledge on distributed computing concepts.
- Must have hands on experience in designing and developing ETL Pipelines in Pyspark in Azure Databricks with strong python scripting exposure like list comprehensions, Dictionary variables etc.
- Must have minimum 5 years of exposure and good proficiency in data warehousing concepts.
- Proficient in SQl and database Design concepts.
- Hands on experience with good proficiency in Delta table and delta file operations like merge, Insert override, Partition overrides etc.
- Hands on Experience in CICD in Azure DevOps/Harness and ADF/Stone branch for orchestration.
- Knowledge of Azure cloud computing platform with Azure Synapse, ADLS.
- Knowledge on GitHub and build management.
- Passion for data and experience working within a data driven organization
- Excellent presentation, communication (oral & written), and relationship building skills, across all levels of management
-
Data Engineer
3 weeks ago
Gurgaon/Gurugram, IN HuQuo Consulting Pvt. Ltd. Full timeJob Description :Role Overview :We are looking for a go-getter person who can define :- Data strategies to meet the demands of business requirements- Technical requirements of the organizational data needs from diverse teams- Data architecture, models and go-forward data solutions for various teams of the organization- The person will be part of the Data...
-
Gurgaon/Gurugram, IN Bigfoot Retail Solutions Pvt. Ltd Full timeJob Description :- Design and build data pipelines from various cloud data sources for the enterprise data warehouse- Partner with Data Engineers, Data architects, domain experts, data analysts and other teams to build foundational data sets that are trusted, well understood, aligned with business strategy and enable self-service- Own and document data...
-
Data Engineer/ETL Developer
3 weeks ago
Gurgaon/Gurugram, IN Huquo Full timeJob Description :- Expertise on SQL, Python, ETL tools, AWS or other cloud- Ability to improve existing ETL pipelines, automate ETL pipelines, and analytical workflows- Experience in the extraction and manipulation of data to conduct analysis to develop understanding on multiple data systems (BI analytics, data blending, data warehousing, data ETL, etc.)- A...
-
Data Engineer
3 weeks ago
Gurgaon/Gurugram, IN HuQuo Consulting Pvt. Ltd. Full timeKey Responsibilities & Skillsets :Common Skillsets :- 5+ years of experience in analytics, Oracle, ETL, Python and associated data engineering jobs.- Must have experience with managing and transforming big data sets using pyspark, spark-scala- Experience with AWS services - Glue, Lambda, EMR, Redshift, RDS- Excellent communication & presentation skills -...
-
Saarthee - Data Engineer - Python/ETL
3 weeks ago
Gurgaon/Gurugram, IN Saarthee Full timePosition Summary : In the Data Engineer role, you will assist on data retrieval from multiple source systems, data processing and cleansing. You will be responsible for constructing automated data pipelines and updating the serving layer for analytics and insights. Your Role Responsibilities and Duties : - Monitor and troubleshoot operational or data issues...
-
Analytics & Data Engineer - SQL/ETL/Python
3 weeks ago
Gurgaon/Gurugram, IN CRESCENDO GLOBAL LEADERSHIP HIRING INDIA PRIVATE L Full timeRole : Analytics and Data Engineer - SQL| Python| ETL, 6+ years, GurgaonWe are seeking an experienced Analytics & Data Engineering professional to join our team and spearhead data initiatives. As a key player in our organization, you will collaborate directly with client leadership to shape the vision and roadmap of their Data, Analytics & Insights team. If...
-
Data Engineer
3 weeks ago
Gurgaon/Gurugram, IN Huquo Consulting Full timeRole : Data EngineerLocation : GurgaonDesignation : AM- AVP (as per interview performance & EXL fitment)Work Timings : 1pm- 10pmRole Overview :We are looking for a go getter person who can define :1. Analytic strategies to meet the demands of business requirements2. Technical requirements of the analytic solutions3. Data requirements of the analytic solution...
-
Data Engineer
1 week ago
Bangalore/Pune/Kolkata/Gurgaon/Gurugram/Mumbai/Ahmedabad, IN Sterlite Technologies Limited Full timeRole as a Data engineer :- Develop and build AI/ML model ready data ponds using Azure Databricks Delta lake- Improve upon the data ingestion models, ETL jobs, and alarm to maintain data integrity and data availability.- Partner with AI/ML product teams proactively identify the data needs for rapid experimentation, stable production deployment and ensure the...
-
Zupee - Senior Data Engineer - ETL/Data Pipeline
3 weeks ago
Gurgaon/Gurugram, IN CASHGRAIL PRIVATE LIMITED Full timeRole & Responsibilities (KRAs) 1) Understand, implement and automate ETL and data pipelines with up-to-date industry standards 2) Hands-on involvement in the design, development and implementation of optimal and scalable data systems 3) Design, build and scale data projects independently and mentor junior data engineers Must - Have Skills: 1) Data Structure...
-
Azure Data Engineer
3 weeks ago
Gurgaon/Gurugram, IN Huquo Full timeJob Description :- To collaborate with various teams/regions in driving facilitating data design, identifying architectural risks and key areas of improvement in data landscape, and developing and refining data models and architecture frameworks- Technical experience and knowledge in Cloud Data Warehousing, data migration and data transformation- Develop and...
-
Senior Data Engineer
1 week ago
Gurgaon/Gurugram, IN Intellismith pvt Ltd Full timeAbout Intellismith :Intellismith, founded in 2019, is a dynamic HR service and technology startup. Our mission is to tackle India's employability challenges head-on. Currently, we operate two key lines of business: recruiting and outsourcing. With teams based in Noida, Chennai, Mumbai, and Bangalore, we collaborate with top brands in the BFSI and IT...
-
Azure Data Engineer
2 weeks ago
Bangalore/Pune/Kolkata/Chennai/Gurgaon/Gurugram, IN VIDPRO CONSULTANCY SERVICES Full timePrimary Roles and Responsibilities:- Developing Modern Data Warehouse solutions using Databricks and Azure Stack - Ability to provide solutions that are forward-thinking in data engineering and analytics space- Collaborate with DW/BI leads to understand new ETL pipeline development requirements.- Triage issues to find gaps in existing pipelines and fix the...
-
Data Engineer
3 weeks ago
Gurgaon/Gurugram/Bangalore, IN Huquo Full timeRole and Responsibilities :- Liaise with different client stakeholders on ad-hoc analyses related to monitoring the entire data lake- Build ETL & data pipelines to help feed the data into different business facing data products/dashboards- Explore options to automate processes & workflows and thus drive efficiencies for client- Work on ML Model based...
-
ETL Developer
3 weeks ago
Gurgaon/Gurugram, IN Sureminds Solutions Pvt. Ltd. Full timeJob Description :1) Have a strong technical background, including 8+ years of experience in designing and engineering large scale systems.2) At least 3 years of experience working as lead in designing, development and deploying data integration and ETL solutions in a data warehousing and BI environment involving data transformations, change data capture, and...
-
Azure Data Engineer
2 weeks ago
Gurgaon/Gurugram, IN Conneqtcorp Full timeRole - Azure Data EngineerLocation - GurugramWork Mode - HybridSkills - Databricks, Azure, Python, SQLNotice Period - May JoinersRole Overview :We are seeking a skilled Data Engineer to join our team. The successful candidate will be responsible for maintaining and optimizing data pipelines, implementing robust data checks, and ensuring the accuracy and...
-
Big Data Developer
3 weeks ago
Gurgaon/Gurugram/Bangalore, IN Workassist.in Full timeOur hiring partner is looking for :- This is a full-time on-site role for a Big Data Developer at Big Four Consulting Firm Gurugram.- The Big Data Developer will be responsible for tasks related to Data Engineering, Extract Transform Load (ETL), Big Data, Data Warehousing, and Software Data Engineering, Big Data, Extract Transform Load (ETL), and Data...
-
AWS Data Engineer/Senior Manager
3 weeks ago
Gurgaon/Gurugram, IN Huquo Full timeJob Description :- Analytic strategies to meet the demands of business requirements- Technical requirements of the analytic solutions- Data requirements of the analytic solution processes- The person will be part of the Data Science team for a major Insurance client. He/ She will work with different stakeholders as SME for data engineering- Engages in...
-
Data Analyst
3 weeks ago
Gurgaon/Gurugram, IN Www.Huquo.com Full timeJob Description :- Participate in the end-to-end delivery lifecycle of data warehousing projects from business requirements gathering to solution design, development, testing and roll-out.- Support the data development lifecycle activities, including data profiling, ETL and data model design and development, testing, and support.- Leverage data analysis...
-
Senior Data Engineer
3 weeks ago
Hyderabad/Chennai/Pune/Mumbai/Gurgaon/Gurugram, IN Hub Work Full timeWe are looking for a data engineer who has experience in building data pipelines for large data platforms. This role will be working with a team building a large data platform catering to the needs of a wealth manager for their application and analytics needs. This role will be augmented with a customer of Intellect and other third parties in delivering...
-
Data Scientist
3 weeks ago
Gurgaon/Gurugram, IN ARAHAS TECHNOLOGIES PRIVATE LIMITED Full timeWe are seeking a skilled Data Scientist with 4-6 years of experience to join our team. The ideal candidate should possess strong analytical skills, a solid understanding of statistical techniques, and hands-on experience with data visualization tools like PowerBI. Additionally, expertise in ETL (Extract, Transform, Load) processes is...