Azure Data Engineer
2 weeks ago
1. Design, develop, and maintain scalable data pipelines and ETL processes using Databricks and PySpark.
2. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and implement data solutions that align with business needs.
3. Optimize and tune existing data pipelines for performance and reliability.
4. Work closely with cloud-based services, particularly Azure or AWS, to deploy and manage data solutions in the cloud environment.
5. Implement data quality and data governance best practices to ensure the accuracy and reliability of the data.
6. Monitor and troubleshoot data pipelines, resolving issues promptly to minimize downtime and data loss.
7. Stay updated with the latest industry trends and technologies related to data engineering,Databricks, PySpark, and cloud platforms.
8. Collaborate with cross-functional teams to integrate data engineering solutions into various applications and systems.
9. Document data engineering processes, architectures, and configurations for reference and knowledge sharing.
10. Provide technical guidance and mentorship to junior data engineers in the team.
Requirements :
1.Design, develop, and maintain scalable data pipelines using Azure Data Factory, Azure Databricks, and other Azure data services.
2. Utilize Databricks for big data analytics and real-time data processing, ensuring optimal performance and efficiency.
3. Collaborate with data architects to design and implement effective data models and architectures to support business requirements.
4. Implement ETL processes for data extraction, transformation, and loading, ensuring the integrity and quality of the data.
5 Integrate data from various sources, ensuring compatibility, accuracy, and adherence to data governance standards.
6. Identify and address performance bottlenecks in data pipelines, optimizing for speed, reliability, and scalability.
7. Work closely with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver effective solutions.
8. Conduct code reviews, enforce best practices in coding and documentation, and mentor junior team members.
Education :
- Bachelor's or master's degree in Computer Science, Information Systems, or a related field.
Experience :
- Minimum of 6 years of hands-on experience as a Data Engineer, with a focus on Azure Data
- Engineering and Databricks.
Technical Skills :
- Proficiency in Azure Data Factory, Azure Databricks, SQL, Python, and ETL processes.
- Solid understanding of data modelling, data warehousing, and big data concepts.
-
Azure Data Engineer
3 weeks ago
Gurgaon/Gurugram, IN HNM Solutions Full timeNote : Immediate joiner's only. 1st, 3rd and 5th Saturdays are workingInterview mode : Walkin Drive interview on 10th-Aug-24 (Saturday) at Gurgaon location (F2F)Role : Azure data engineeringLocation : GurgaonExperience : 3+ yearsMandatory :- DataBricks Engineer, SQL Engineer and a Mongodb Engineer- Little with any bid data exp (Hands on)Roles and...
-
Azure Data Engineer
3 weeks ago
Gurgaon/Gurugram, IN TETRAHED INC Full timeRoles and Responsibilities :Azure Data Engineering :- Design, develop, and maintain data pipelines using Azure Data Factory, Azure Databricks, and other Azure data services.- Implement data storage solutions leveraging Azure SQL Database, Azure Data Lake, and other relevant Azure services.- Ensure data integration and ETL processes are optimized for...
-
Data Engineer
3 weeks ago
Gurgaon/Gurugram, IN Elite recruitments Full timeJob opportunity for data engineers for Gurgaon location. Experience : 8+ years.Job Description :Role & Responsibilities :- Should have good experience in databricks.- A seasoned data team member with a good grasp of both business and technical aspects.- Familiarity with data stack.- Interact with the other members of the data governance team, business teams...
-
Azure Data Engineer
2 weeks ago
Gurgaon/Gurugram/Bangalore/Chennai/Pune/Mumbai, IN Element Infomatics Pvt. Ltd. Full timeAbout the job :Company Description:Element Technologies Inc is a technology company based in Bangalore, committed to offering high-value IT services using flexible engagement and delivery models. We collaborate with global industry-leading technologies to deliver greater business value in sectors such as Life Sciences, Banking & Financial Services, Retail,...
-
Affine - Lead Data Engineer - Azure Platform
2 weeks ago
Gurgaon/Gurugram, IN Affine Analytics Pvt Ltd Full timeRoles and Responsibilities :- Resource and Project Timeline Estimation : Accurately estimate project timelines and resource allocation for Azure-based cloud projects.- Defining Project and Program Management Process : Establish project and program management frameworks for Azure environments.- Task Prioritization and Resource Management : Allocate tasks...
-
Lead Data Engineer
3 weeks ago
Gurgaon/Gurugram, IN StorySideUp Full timeJob Description : We are seeking a passionate, hands-on and experienced Lead Data Engineer to join our Databricks Center of Excellence. You will play a key role in helping customers understand and implement Databricks Lakehouse platform, and to achieve their business goals. The candidate will collaborate with client stakeholders on multiple projects in Big...
-
India Infoline
3 weeks ago
Gurgaon/Gurugram, IN India Infoline Full timeKey Responsibilities :- Data Lakehouse Development : Lead the design and implementation of our data lakehouse architecture using Azure and Databricks.- Data Modeling : Develop conceptual, logical, and physical data models utilizing star schema methodologies.- Pipeline Optimization : Build and optimize data pipelines using PySpark and Databricks, ensuring...
-
Strawberry Infotech
3 weeks ago
Gurgaon/Gurugram, IN Strawberry Infotech Pvt. Ltd Full timePosition : Data Engineer Type : Full-time Hybrid Location : Gurugram Experience Required : - 4+ years As a Data Engineer at Strawberry InfoTech, you will play a vital role in designing and implementing data solutions that drive business intelligence and analytics.- You will collaborate with cross-functional teams to ensure data availability and performance,...
-
Azure Databricks Engineer
2 days ago
Gurgaon/Gurugram, IN Arrixa Full timeKey Responsibilities :- Data Pipeline Development : Design, develop, and optimize data pipelines and workflows in Azure Databricks for ETL/ELT processing, data transformation, and analytics. - Big Data Solutions : Architect and implement big data solutions using Azure Databricks, Spark, Azure Data Lake, and other Azure data services. - Data Integration :...
-
Data Engineer
1 week ago
Mumbai/Bangalore/Pune/Chennai/Hyderabad, IN Changeleaders Full timeCompany Overview :Tech Mahindra is a leading provider of digital transformation, consulting, and business re-engineering services and solutions. We leverage next-generation technologies to help our clients achieve their business objectives.Job Summary :We are looking for an experienced Azure Data Engineer to join our dynamic team. The ideal candidate will be...
-
Senior Azure Data Engineer
3 weeks ago
Gurgaon/Gurugram, IN Gallop Research Full timeRoles and Responsibilities : - Build ETL & data pipelines using Azure Data Factory & Databricks to help feed the data into data products/dashboards. - Automate processes and workflows to drive efficiencies for client. - Liaise with different stakeholders on ad-hoc analyses and monitor the entire DWH/Data Lake. - Working with stakeholders to gather...
-
Senior Azure Data Engineer
3 weeks ago
Gurugram, IN Kool KodersOWOW Full timeKey Responsibilities : Design and Development : - Design, develop, and optimize large-scale data processing pipelines using Databricks on Azure.- Implement ETL processes to ingest and transform data from various sources into our data platform.- Develop scalable and efficient data models to support analytics and reporting needs. Azure Integration : -...
-
Data Engineer
3 weeks ago
Gurgaon/Gurugram, IN curatAId Full timeJob Description :Expertise in Azure cloud tools for our client, a global business and technology services firm. In this role, you will be integral to designing, developing, and maintaining the data infrastructure. The ideal candidate will have extensive experience with cloud-based data technologies, particularly in Azure, and a proven track record of...
-
Azure Data Engineer
3 weeks ago
Anywhere in India/Multiple Locations/Mumbai/Bangalore/Hyderabad/Kolkata/Chennai/Pune/Bhubaneshwar/Na, IN Talpro Full timeJob Description : We are looking for a skilled and experienced Azure Data Engineer with expertise in ADF (Azure Data Factory), strong RDBMS and SQL skills, and familiarity with Informatica PWC and Oracle PL/SQL. As an Azure Data Engineer, you will be responsible for understanding and converting Informatica PWC and Oracle PL/SQL code into ADF and Azure SQL....
-
Lead Azure Data Engineer
3 months ago
Navi Mumbai/Pune/Bangalore, IN Techno Wise Full timeJob Summary : We are seeking a highly skilled and motivated individual to join our team as a Senior Azure Data Engineer. The ideal candidate will have a strong background in Azure data technologies, a proven track record of ability to design and implement scalable and efficient data solutions.Responsibilities :- Design, develop, and deploy data pipelines and...
-
Big Data Engineer
3 weeks ago
Hyderabad/Pune/Delhi NCR, IN Ace Recruit Full timeJob Description :As a Big Data Engineer (Azure), you will build and learn about a variety of analytics solutions & platforms, data lakes, modern data platforms, data fabric solutions, etc. using different Open Source, Big Data, and Cloud technologies on Microsoft Azure. On a typical day, you might :- Design and build scalable & metadata-driven data ingestion...
-
Data Engineer
5 months ago
Bangalore/Pune, IN OptOut Full timeRole Description :- Data modeling and architecture : You would be responsible for designing and implementing data models that are optimized for Azure services, including Azure SQL Database, Azure Cosmos DB, and Azure Data Lake Storage.- ETL and data integration: You would be responsible for building and maintaining data pipelines that move data from various...
-
Azure Data Engineer
2 weeks ago
Gurgaon/Gurugram, IN Deqode Full timeJob Description : - Design and build data pipelines using Spark-SQL and PySpark in Azure Databricks- Design and build ETL pipelines using ADF- Build and maintain a Lakehouse architecture in ADLS / Databricks.- Perform data preparation tasks including data cleaning, normalization, deduplication, type conversion etc.- Work with DevOps team to deploy solutions...
-
Data Engineer
3 weeks ago
Gurgaon/Gurugram, IN Drimlite Full timeJob Title : Data EngineerLocation : Gurugram, DelhiExperience : 3-5 YearsBudget : 24 LPAWork Mode : Work From Office (WFO)Interview Mode : VirtualMust-Have Skills : PySpark, Databricks, ML Flow, Azure DevOps, SQLJob Description :We are seeking a skilled Data Engineer to design, build, and maintain data pipelines, data architectures, and ETL processes using...
-
Data Engineer
3 weeks ago
Delhi NCR/Gurgaon/Gurugram, IN Drimlite Full timeJob Description :- Design and build data pipelines using Spark-SQL and PySpark in Azure DatabricksJob Description :- Design and build ETL pipelines using ADF- Build and maintain a Lakehouse architecture in ADLS / Databricks.- Perform data preparation tasks including data cleaning, normalization, deduplication, type conversion etc.- Work with DevOps team to...