Senior Data Engineer
2 weeks ago
Senior Data Engineer (Sr. Developer)
- Proficient in Azure Databricks, Azure Data Factory and Azure SQL DB
- Proficient in designing and developing ingestion pipelines with DLT and Structured Streaming
- Proficient in Databricks/Spark internals
- Proficient in Spark jobs optimization techniques
- Proficient in designing the pipelines with latest Databrick/Spark optimization techniques
- Proficient in PySpark, SQL and Python for data engineering
- Experience in developing and maintaining meta data driven data ingestion framework
- Extensive experience in the data warehousing concepts
- Certification in professional data bricks is preferred
- Exhibits leadership skills
The Senior Data Engineer will be responsible for building and maintaining robust, scalable, and efficient data pipelines that support our data-driven initiatives.
You will work alongside data scientists, analysts, and other engineering teams to design, develop, and optimize data processing and transformation workflows.
As a Senior Data Engineer, you will also ensure the reliability, performance, and scalability of data architectures while mentoring junior engineers and contributing to the development of best practices.
Key Responsibilities :
- Develop end-to-end data pipelines to ingest, transform, and store large-scale datasets using technologies like Azure Data Factory, Databricks, and SQL.
- Work closely with data engineers, analysts, and business stakeholders to understand data needs and deliver solutions that support data-driven decision-making.
- Develop data ingestion pipelines utilizing Delta Live Tables (DLT) and Structured
- Streaming to handle both batch and real-time data ingestion requirements.
- Continuously optimize the performance and scalability of data pipelines, ensuring data is processed efficiently and in a timely manner.
- Implement processes to ensure the integrity, accuracy, and consistency of data, including monitoring and resolving data quality issues.
- Troubleshoot and resolve data pipeline issues, set up monitoring systems, and proactively address performance or data-related problems.
- Document data engineering processes, workflows, and standards.
- Contribute to the development and adoption of best practices within the team.
- Keep abreast of the latest data engineering technologies, methodologies, and trends to continuously improve and refine data engineering :
- Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field.
- 5+ years of experience in data engineering, building and managing data pipelines, and working with large datasets.
- Proficiency in working with Azure Data Factory, Databricks, and SQL-based data solutions.
- Strong experience in data pipeline development, data processing, and transformation using Spark Structured Streaming and Delta Live Tables.
- Familiarity with cloud-based data storage solutions (e.g., Azure Blob Storage, Azure SQL DB, or similar platforms).
- Solid understanding of ETL/ELT processes and data warehousing concepts.
- Azure Data Engineer, or related certifications, are preferred but not mandatory.
- Strong programming skills in Python, SQL, and other relevant data engineering languages.
- Experience with data orchestration tools like Apache Airflow or Azure Data Factory.
- Proficiency in using version control systems (e.g., Git) and CI/CD pipelines for data solutions.
- Strong analytical skills, with the ability to troubleshoot complex data issues and optimize performance across data systems.
- Excellent teamwork and communication skills, with the ability to work with cross-
- functional teams and communicate complex technical concepts to non-technical stakeholders.
-
Senior Data Engineer
1 day ago
India MindBrain Full timeJob Title : Senior Data Engineer (Informatica, IICS, Snowflake, SQL) Location : Remote Experience : 5 years Job Description : We are seeking a skilled Senior Data Engineer with expertise in Informatica, IICS, Snowflake, and SQL to join our team. The ideal candidate should have experience in designing, developing, and maintaining data pipelines and ETL...
-
Senior Data Engineer
4 weeks ago
india, india Ampstek Full timeTitle: Senior Data Engineer Location: Remote – India Job Type: Fulltime Employee Overlap: 4 hours with PST (Pacific Standard Time) or E Availability: Looking for Immediate Joiner or by max 1 Week only Detailed Job Description: • Candidate proficiency level: Senior (8+) Python and SQL proficiency along with use of Amazon Redshift for Data Warehousing ,...
-
Senior Big Data Developer
14 hours ago
India ENABLE DATA TECHNOLOGIES PRIVATE LIMITED Full time**Job Overview:**We're seeking an experienced Senior Spark/Java Developer to join our team at Enable Data Technologies Private Limited.This role involves designing and developing large-scale data processing systems using Spark-based frameworks, collaborating with cross-functional teams to enhance data platforms and service-oriented architectures, and...
-
Data Engineering Lead
18 hours ago
India ENABLE DATA TECHNOLOGIES PRIVATE LIMITED Full time**Job Description:**We're looking for a seasoned Senior Spark/Java Developer to join our team at Enable Data Technologies Private Limited.This role requires a deep understanding of software engineering principles, particularly in designing and developing large-scale data processing systems using Spark-based frameworks.As a member of our team, you'll work...
-
Senior Data Engineer
1 day ago
India Mytat Rvr Innovation Full timeRole : Data Engineer Location : Gurugram / Remote Notice period : Immediate-30 Days (preferred) Experience : 8 Years We are currently hiring Senior Data Engineers on Mytat's payroll, where you will have the opportunity to work with established industry leaders. The compensation will be aligned with industry standards. Requirements : - Experience in Data/SQL...
-
Senior Data Engineer
3 hours ago
India brightedge Full timeAbout the RoleWe are looking for a highly motivated and experienced Senior Data Engineer to design, implement, and optimize data collection, storage, and processing pipelines. The ideal candidate will have expertise in handling structured and unstructured data, optimizing query performance, and ensuring efficient data retrieval.
-
Senior Data Engineer
10 hours ago
India Hunarstreet Technologies Full timeKey Requirements : - Technical Skills : Databricks, Python, Scala (mandatory), Spark, SQL, Lakehouse, AWS or Azure, OpenSearch - Experience : Must have hands-on expertise and be able to contribute from Day 1 - Education : Engineering degree required - Communication : Strong communication skills - Work Style : Individual contributors who can also lead a team...
-
Senior Data Engineer
10 hours ago
India Apidel Technologies Full timeSenior Data Engineer Key Responsibilities : - Design, develop, and implement data pipelines and data processing solutions using Python/PySpark. - Perform data modeling, data analysis, and data transformation tasks. - Utilize Azure Databricks, Azure Data Factory, Azure Data Lake, and Azure DevOps to build and deploy data solutions. - Ensure data quality and...
-
Senior Data Engineer
9 hours ago
India Iclivia Full timeRole : Senior Data : 5 Years Location : Remote Employment Type : Full-time Job Summary : We are seeking a highly skilled Senior Data Engineer with 5 years of experience in SQL, Python, .NET, SSIS, and SSAS, along with 2 years of experience in Azure Cloud and CI/CD deployment using Git. The ideal candidate will be responsible for designing, developing, and...
-
Senior Data Architecture Engineer
18 hours ago
India Pylon Management Consulting Full timeWe are looking for an experienced Senior Data Architecture Engineer to lead our data engineering efforts and design innovative data architecture solutions. In this role, you will be responsible for monitoring, optimizing, and maintaining data workflows to ensure high performance, reliability, and scalability of integration processes.Key Responsibilities:Data...
-
Senior Data Engineer
4 weeks ago
india Right Advisors Private Limited Full timeJob Description:Senior Data EngineerWe are seeking a highly skilled and motivated Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in data engineering, with a strong focus on Databricks, Python, and SQL. As a Data Engineer you will play a crucial role in designing, developing, and maintaining our data infrastructure...
-
Senior Data Engineer
2 weeks ago
India Mytat Rvr Innovation Full timeRole : Data Engineer Location : Gurugram / RemoteNotice period : Immediate-30 Days (preferred)Experience : 8 YearsWe are currently hiring Senior Data Engineers on Mytat's payroll, where you will have the opportunity to work with established industry leaders. The compensation will be aligned with industry standards.Requirements :- Experience in Data/SQL-...
-
Senior Azure Data Engineer
4 weeks ago
india Zensoftit solutions Full timeRole DescriptionThis is a full-time hybrid role for a Senior Azure Data Engineer at Zensoft IT Solutions with Client Wynn. The role is based in Hyderabad with some work-from-home flexibility. The Senior Azure Data Engineer will be responsible for data engineering, data modelling, ETL processes, Snowflake, Python, data warehousing, and data analytics on Azure...
-
Senior Data Engineer
4 weeks ago
india InCommon Full timeSenior Data EngineerJob Type:Full-TimeLocation:Bangalore, Hybrid (Startup Environment)⏱️Work Timings:Flexible️Experience Required:5+ years in data engineering, with expertise in scalable data platformsHiring Process:Resume review - Initial screen - Technical and Cultural Interviews - OfferAbout the CompanyInCommon is hiring on behalf of Saltmine....
-
Senior Data Engineer
1 week ago
India Lingaro Full timeJob DescriptionJob Title: Azure Data Engineer- Senior ConsultantAbout Lingaro:Lingaro Group is the end-to-end data services partner to global brands and enterprises. We lead our clients through their data journey, from strategy through development to operations and adoption, helping them to realize the full value of their data.Since 2008, Lingaro has been...
-
Senior Data Engineer
3 weeks ago
India Mogra Ai Full timeImmediate Joining | 3 Months Probation | Salary: ₹14 LPALocation: RemoteJob Type: Full-Time / ContractExperience Level: Mid-Senior/SeniorJob Summary:We are looking for a highly skilled Senior Data Engineer to design, develop, and manage data pipelines, implement OpenTelemetry (OTEL) for observability, and create actionable visualizations. The ideal...
-
Senior Data Engineer
10 hours ago
India Techmynd Consulting Full timeRole : Senior Data : Remote, IND Experience : 4-7 Must-Have : - Experience with Snowflake, ETL, and Python. - Experience with Apache, and : - Design, develop, and maintain robust and scalable data pipelines for ELT and ETL processes, ensuring data accuracy, completeness, and timeliness. - Work with stakeholders to understand data requirements and translate...
-
Senior Data Engineer
4 weeks ago
india PRI Global Full timeHiring Senior Data Engineer with 7+ years of experience for the Hyderabad location, Hybrid Mode.Job Description:- Proficient across Azure Data factory and understands Change data capture- Familiar with Change management practices (branches, pull requests, config transformations) and use of CI/CD- Familiar with Databricks & Unity catalog- Proficient in SQL...
-
Senior Data Engineer Position
22 hours ago
India Mytat Rvr Innovation Full timeCompany Overview">We are Mytat Rvr Innovation, a leader in the industry. Our company is constantly innovating and expanding its services.">About This Job">We're seeking an experienced Senior Data Engineer to join our team. As a key member of our data engineering group, you will have the opportunity to work with a talented team on a wide range of...
-
Senior Azure Data Engineer
4 weeks ago
india Zensoftit solutions Full timeRole DescriptionThis is a full-time hybrid role for a Senior Azure Data Engineer at Zensoft IT Solutions with Client Wynn. The role is based in Hyderabad with some work-from-home flexibility. The Senior Azure Data Engineer will be responsible for data engineering, data modelling, ETL processes, Snowflake, Python, data warehousing, and data analytics on Azure...