Data Engineer

4 weeks ago

Delhi, India Erobern Consulting Full time
Mandatory Skills – Pyspark / Databricks, Cloud (Azure or AWS), Big Data, SQL, Nice to have to Unity catalog

Job Title: Azure Databricks Lead

Need Immediate joiner

Job Overview:As an

Azure Databricks Lead/Specialist , you will play a critical role in designing, implementing, and optimizing data solutions using Azure Databricks. Your expertise will contribute to building robust data pipelines, ensuring data quality, and enhancing overall performance. You'll collaborate with cross-functional teams to deliver high-quality solutions aligned with business requirements.Responsibilities:1.

Design and Develop Data Pipelines :o Create scalable data processing pipelines using

Azure Databricks

and

PySpark .o Implement ETL (Extract, Transform, Load) processes to ingest, transform, and load data from various sources.o Collaborate with data engineers and architects to ensure efficient data movement and transformation.2.

Data Quality Implementation :o Establish data quality checks and validation rules within Azure Databricks.o Monitor data quality metrics and address anomalies promptly.o Work closely with data governance teams to maintain data accuracy and consistency.3.

Unity Catalog Integration :o Leverage

Azure Databricks Unity Catalog

to manage metadata, tables, and views.o Integrate Databricks assets seamlessly with other Azure services.o Ensure proper documentation and organization of data assets.4.

Delta Lake Expertise :o Understand and utilize

Delta Lake , which provides ACID transactions and time travel capabilities on top of data lakes.o Implement Delta Lake tables for reliable data storage and versioning.o Optimize performance by leveraging Delta Lake features.5.

Performance Tuning and Query Optimization :o Profile and analyze query performance.o Optimize SQL queries, Spark jobs, and transformations for efficiency.o Tune resource allocation to achieve optimal execution times.6.

Resource Optimization :o Manage compute resources effectively within Azure Databricks clusters.o Scale clusters dynamically based on workload requirements.o Monitor resource utilization and cost efficiency.7.

Source System Integration :o Integrate Azure Databricks with various source systems (e.g., databases, data lakes, APIs).o Ensure seamless data ingestion and synchronization.o Handle schema evolution and changes in source data.8

Stored Procedure Conversion in Databricks:· Convert existing stored procedures (e.g., from SQL Server) into Databricks-compatible code.· Optimize and enhance stored procedures for better performance within Databricks· SSRS conversion experience

Qualifications and Skills:·

Education : Bachelor's degree in Computer Science, Information Technology, or a related field.·

Experience :o Minimum 5 years of experience architecting and building data platforms on

Azure .o Proficiency in

Azure Databricks ,

PySpark , and

SQL .o Familiarity with

Delta Lake

concepts.·

Certifications

(preferred):o

Microsoft Certified: Azure Data Engineer Associate

or similar.o

Databricks Certified Associate Developer for Apache Spark .·

Soft Skills :o Strong problem-solving abilities.o Excellent communication and collaboration skills.o Ability to work in a fast-paced, agile environment.oAdditional Considerations:·

Forward-Thinking Approach :o As an Azure Databricks Specialist, you'll contribute to innovative solutions that drive business insights and enhance data capabilities.o Stay updated with industry trends and emerging technologies.

  • Security Engineer L1

    2 weeks ago

    Delhi, Delhi, India NTT DATA Full time

    Job Title: Security Engineer L1About the RoleNTT DATA is seeking a highly skilled Security Engineer L1 to join our team. As a Security Engineer L1, you will be responsible for providing a managed service to clients to ensure that their Firewall infrastructure remains operational.Key ResponsibilitiesProactively identify, investigate, and route incidents to...

  • Senior Data Engineer

    5 days ago

    Chennai/Gurgaon/Gurugram/Delhi NCR/Hyderabad, India Eucloid Data Solutions Pvt Ltd Full time

    About Eucloid Data SolutionsEucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. We offer bespoke, full stack and data-driven approach to solve highly complex and large-scale business problems for some of the biggest Fortune 500...

  • Senior Data Engineer

    5 days ago

    Chennai/Gurgaon/Gurugram/Delhi NCR/Hyderabad, India Eucloid Data Solutions Pvt Ltd Full time

    About the Role:Eucloid Data Solutions Pvt Ltd is seeking a highly skilled Senior Data Engineer to join our Data Platform team. The successful candidate will have hands-on expertise in Databricks and will be responsible for designing and building data infrastructure to support various business applications.Key Responsibilities:Design, implementation, and...

  • Senior Cybersecurity Engineer

    11 hours ago

    Delhi, Delhi, India NTT DATA Full time

    About the RoleWe are seeking a skilled Security Managed Services Engineer to join our team. As a seasoned engineer, you will be responsible for providing proactive incident resolution and technical problem-solving to our clients.Your Key ResponsibilitiesConduct daily threat hunting to identify advanced threats.Analyze log sources to identify potential...

  • Data Engineer

    1 week ago

    Delhi, India Implere Technolgies Pvt Ltd Full time

    Role DescriptionThis is a full-time remote role for a Data Engineer at Implere Technologies Pvt Ltd. The Data Engineer will be responsible for tasks such as data engineering, data modeling, ETL processes, data warehousing, and data analytics on a day-to-day basis.QualificationsData Engineering and Data Modeling skillsExperience in ETL (Extract Transform...

  • Data Engineer

    1 month ago

    delhi, India Zemoso Technologies Full time

    Experience: 4.5-6 Years data pipelines experience Key Responsibilities: Python Proficiency: Demonstrate a strong command of Python programming language, actively contributing to the development and maintenance of data engineering solutions. Data Engineering Expertise: Set up and maintain efficient data pipelines, ensuring smooth data flow and integration...

  • Data Engineer

    1 week ago

    Delhi, India Implere Technolgies Pvt Ltd Full time

    Role DescriptionThis is a full-time remote role for a Data Engineer at Implere Technologies Pvt Ltd. The Data Engineer will be responsible for tasks such as data engineering, data modeling, ETL processes, data warehousing, and data analytics on a day-to-day basis.QualificationsData Engineering and Data Modeling skillsExperience in ETL (Extract Transform...

  • Data Engineer

    1 month ago

    delhi, India Zemoso Technologies Full time

    Experience: 4.5-6 Years data pipelines experienceKey Responsibilities:Python Proficiency:Demonstrate a strong command of Python programming language, actively contributing to the development and maintenance of data engineering solutions.Data Engineering Expertise:Set up and maintain efficient data pipelines, ensuring smooth data flow and integration across...

  • Service Delivery Field Support Engineer

    2 weeks ago

    Delhi, Delhi, India NTT DATA Full time

    Job OverviewNTT DATA is seeking a highly motivated and detail-oriented Service Delivery Field Support Engineer to join our team. As a key member of our support team, you will be responsible for providing technical support to clients, ensuring that their infrastructure is configured, installed, and operational.Key Responsibilities:Ensure that client...

  • Senior Data Engineer

    2 weeks ago

    Gurgaon/Gurugram/Chennai/Hyderabad/Delhi NCR, India Eucloid Data Solutions Pvt Ltd Full time

    About Eucloid Data Solutions Pvt LtdEucloid Data Solutions Pvt Ltd is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Our company offers bespoke, full stack and data-driven approach to solve highly complex and large-scale business problems for some of the...

  • Data Engineer

    2 weeks ago

    Delhi, India Ryden Global Full time

    We're #hiring #DataEngineers for our USA based client at offshore in India (must support afternoon shift in IST).Role DescriptionThis is a full-time/contract remote role for a Data Engineer at Ryden Global. The Data Engineer will be responsible for data engineering tasks such as data modeling, ETL processes, data warehousing, and data analytics on a...

  • Senior Data Engineer

    2 weeks ago

    Chennai/Gurgaon/Gurugram/Delhi NCR/Hyderabad, India Eucloid Data Solutions Pvt Ltd Full time

    About Eucloid Data SolutionsEucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning, and Data engineering. We offer a bespoke, full-stack, and data-driven approach to solve highly complex and large-scale business problems for some of the biggest Fortune 500...

  • MS Engineer L3

    2 weeks ago

    Delhi, Delhi, India NTT DATA Full time

    Job DescriptionMake a Difference with NTT DATANTT DATA is a leading global innovator of business and technology services, serving 75% of the Fortune Global 100. We're committed to helping clients innovate, optimize, and transform for long-term success. As a seasoned Managed Services Cross Technology Engineer (L3), you'll play a critical role in providing...

  • Data Engineer

    2 weeks ago

    Delhi, Delhi, India CodeKart Full time

    Job Title: Data EngineerAt CodeKart, we are seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems and pipelines.Key Responsibilities:Design and implement data pipelines and ETL processes using AWS services.Develop and maintain data warehouses...

  • Data Engineer

    2 weeks ago

    Delhi, Delhi, India Amossys Consulting Services Full time

    Job Title: Data EngineerWe are seeking a skilled Data Engineer to join our team at Amossys Consulting Services. The ideal candidate will have hands-on experience in PySpark and Big Data platforms, with a strong focus on building scalable data pipelines and working with cloud-native services.Key Responsibilities:Develop and optimize data pipelines using...

  • Data Engineer

    2 weeks ago

    Delhi, Delhi, India CodeKart Full time

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at CodeKart. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems and architectures.Responsibilities:Design and implement data pipelines and architectures using AWS services such as EMR, Glue, Lambda, SNS,...

  • Data Engineer

    4 weeks ago

    delhi, India Recro Full time

    Job Title: Azure Data Engineer Location: Remote Experience: 4+ years Job Description: We are seeking a skilled Azure Data Engineer with expertise in Python, SQL, data modeling, and Azure technologies to join our team. The ideal candidate will have experience developing business data models and products, with a strong focus on data flows from ingestion to...

  • Data Engineer

    3 weeks ago

    Delhi, India EXL Full time

    Company DescriptionAt EXL, we focus on collaboration and tailoring solutions to meet the unique needs of our clients. Our expertise in transformation, data science, and change management helps businesses make better decisions and drive intelligence into their digital operations. We specialize in analytics, digital interventions, and operations management to...

  • Senior Data Engineer

    2 weeks ago

    Chennai/Gurgaon/Gurugram/Delhi NCR/Hyderabad, India Eucloid Data Solutions Pvt Ltd Full time

    About Eucloid Data Solutions Pvt LtdEucloid Data Solutions Pvt Ltd is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Our company offers bespoke, full-stack and data-driven approach to solve highly complex and large-scale business problems for some of the...

  • Senior IT Engineer

    2 weeks ago

    Delhi, Delhi, India NTT DATA Full time

    About the RoleNTT DATA is seeking a highly skilled Senior MS Engineer to join our team. As a key member of our managed services team, you will be responsible for providing technical support and guidance to our clients, ensuring the smooth operation of their IT infrastructure and systems.Key ResponsibilitiesProactively monitor and manage IT infrastructure and...