Senior Data Engineer
1 month ago
As a Senior Data Engineer at Expian Technologies Pvt Ltd, you will be responsible for designing, implementing, and maintaining data processing pipelines and workflows using Databricks on the Azure platform. Your expertise in PySpark, SQL, Databricks, test-driven development, and Docker will be essential to the success of our data engineering initiatives.
Key Responsibilities
• Collaborate with cross-functional teams to understand data requirements and design scalable and efficient data processing solutions.
• Develop and maintain data pipelines using PySpark and SQL on the Databricks platform.
• Optimize and tune data processing jobs for performance and reliability.
• Implement automated testing and monitoring processes to ensure data quality and reliability.
• Work closely with data scientists, data analysts, and other stakeholders to understand their data needs and provide effective solutions.
• Troubleshoot and resolve data-related issues, including performance bottlenecks and data quality problems.
• Stay up to date with industry trends and best practices in data engineering and Databricks.
Requirements
• 5+ years of experience as a Data Engineer, with a focus on Databricks and cloud-based data platforms with a minimum of 2 years of latest experience with TDD on Databricks.
• Hands-on experience in PySpark programming for data manipulation, transformation, and analysis.
• Strong experience in SQL and writing complex queries for data retrieval and manipulation.
• Experience in developing and implementing test cases for data processing pipelines using a test-driven development approach.
• Experience in Docker for containerising and deploying data engineering applications.
• Experience in the scripting language Python is mandatory.
• Strong knowledge of Databricks platform and its components, including Databricks notebooks, clusters, and jobs.
• Experience in designing and implementing data models to support analytical and reporting needs will be an added advantage.
• Strong Knowledge of Azure Data Factory for Data orchestration, ETL workflows, and data integration is good to have.
• Good to have knowledge of cloud-based storage such as Amazon S3 and Azure Blob Storage.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• Strong analytical and problem-solving skills.
• Strong English communication skills, both written and spoken, are crucial.
• Capability to solve complex technical issues and comprehend risks prior to the circumstance.
-
Senior Data Engineering Specialist
1 month ago
Bengaluru, Karnataka, India NTT DATA Full timeAbout the RoleWe are seeking a skilled Senior Data Engineering Specialist to join our team in Bangalore, India. The successful candidate will have a strong background in ETL development and data engineering using Azure concepts and technologies.Key ResponsibilitiesDesign and implement high and low-level design documents for ETL processes.Develop ETL code in...
-
Data Engineering Senior Specialist
1 month ago
Bengaluru, Karnataka, India NTT DATA Services Full timeData Engineering Senior Specialist RoleDiverse and dynamic Data Engineering activities on Azure Data platform. The role involves building and delivering Data pipelines connecting various enterprise data sources, both RDBMS and NoSQL. Develop data mappings to existing data sources and design big data processing notebooks using Azure Databricks. Scripting and...
-
Senior Data Engineering Manager
2 days ago
Bengaluru, Karnataka, India Data Labs - LANDMARK GROUP Full timeAbout the RoleWe are seeking a Senior Data Engineering Manager to lead our engineering team in successfully building and deploying cutting-edge analytics products for consumption by various stakeholders.As a key member of our team, you will be responsible for leading the architecture, design, and delivery of enterprise-grade microservices. You will also...
-
Bengaluru, Karnataka, India NTT DATA Full timeJob Title: Senior Data Engineer for AI and Machine LearningWe are seeking an experienced Data Scientist & AIML, GenAI Lead/Engineer to join our team in Bangalore, India.About the Role:Develop and implement traditional machine learning algorithms.Deploy at least one model in a production environment.Write and maintain Python code for data science and machine...
-
Senior Time Series Data Engineer
3 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeAbout the RoleWe are seeking a highly skilled Senior Time Series Data Engineer to join our team in Bangalore, India. As a Senior Time Series Data Engineer, you will be responsible for designing, developing, and implementing large-scale data processing systems that handle complex time series data.Key ResponsibilitiesDesign and develop scalable data pipelines...
-
Senior AI Data Engineer and Solutions Architect
3 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeAbout NTT DATANTT DATA is a global innovator of business and technology services, serving 75% of the Fortune Global 100. Our diverse experts in over 50 countries collaborate to drive long-term success for clients.Job OverviewWe are seeking an experienced Senior AI Data Engineer and Solutions Architect to join our team in Bangalore, India. This role requires...
-
Senior Data Architect
2 weeks ago
Bengaluru, Karnataka, India NTT DATA Services Full timeJob OverviewWe are seeking an experienced Senior Data Architect - Cloud Engineer to join our team in Remote India. The successful candidate will play a critical role in supporting and enhancing our Azure Data Lakehouse environment.About the RoleThis is an exciting opportunity to leverage your expertise in designing and implementing efficient data models,...
-
Senior Cloud Data Architect
3 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeNTT DATA is a global technology services company with a presence in over 50 countries.We have a wide range of job opportunities available for talented professionals like you.About the Job:Job DescriptionWe are seeking a Senior Cloud Data Architect to join our team in Bangalore, India. The successful candidate will be responsible for designing and...
-
Senior Power BI Data Engineer Professional
2 weeks ago
Bengaluru, Karnataka, India NTT DATA Services Full timeJob OverviewWe are seeking a highly skilled Senior Power BI Data Engineer to join our team in Bangalore, India. This role will involve developing and implementing data engineering solutions using Power BI reporting tools.About the RoleThis is an exciting opportunity for an experienced data engineer to work with our team and contribute to the development of...
-
Snowflake Data Engineer and Architect
3 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeAbout the RoleWe are seeking a highly skilled Snowflake Data Engineer and Architect to join our team in Bangalore, India. This is a senior-level position that requires expertise in designing, developing, and maintaining large-scale data warehousing solutions using Snowflake.Key ResponsibilitiesDesign and implement efficient data models and architectures for...
-
Senior Engineering Manager
1 month ago
Bengaluru, Karnataka, India HEVO DATA Full timeJob Description:This is an excellent opportunity to lead the development of scalable and fault-tolerant platform services at Hevo Data.As a Senior Engineering Manager, you will be responsible for building and managing a team of passionate software engineers who share a sense of ownership and impact.The successful candidate will drive on-time, high-quality...
-
Senior Voice Systems Engineer
1 week ago
Bengaluru, Karnataka, India NTT DATA Full timeJob OverviewWe are seeking a seasoned Senior Voice Systems Engineer to join our team in Bangalore, India.About the RoleThis is a critical position that requires expertise in design, deployment, and support of voice platforms. As a Senior Voice Systems Engineer, you will be responsible for managing business-driven programs and projects across voice products...
-
Senior Cloud Data Architect
3 weeks ago
Bengaluru, Karnataka, India NTT DATA Services Full timeAbout the RoleWe are seeking an experienced Senior Cloud Data Architect to join our team at NTT DATA Services. As a key member of our data engineering team, you will be responsible for designing and implementing cloud-based data solutions on Azure Data platform.ResponsibilitiesDesign and develop big data processing notebooks using Azure DatabricksScripting...
-
Data Engineering Expert
3 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeAbout the RoleWe are seeking a skilled Data Engineer to join our team in Bangalore, India. As a key member of our data engineering team, you will play a crucial role in designing and implementing automated data pipelines.
-
Senior Data Solutions Consultant
1 day ago
Bengaluru, Karnataka, India HEVO DATA Full timeOverview: We're seeking a skilled Senior Data Solutions Consultant to join our Sales team. This role involves working closely with the sales team to understand customers' business and technical needs and demonstrating how Hevo's platform can address their real-time data replication requirements.
-
Senior Quality Assurance Engineer
2 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeAbout the Role">NTT DATA Services is a leading provider of IT and business services, with a strong presence in global markets. As part of our team, you will have the opportunity to work on cutting-edge projects that drive innovation and growth.Job OverviewWe are currently seeking a skilled Senior Quality Assurance Engineer to join our team in Bangalore,...
-
Senior Data Architect
3 weeks ago
Bengaluru, Karnataka, India NTT DATA Services Full timeAbout the RoleNTT DATA Services is seeking a highly skilled Senior Data Architect to join our team in Bangalore, India.The Ideal CandidateThis is an exceptional opportunity for an experienced data professional to work with cutting-edge technologies and contribute to the design and development of innovative solutions. To be successful in this role, you will...
-
Bengaluru, Karnataka, India NTT DATA Full timeAbout the RoleWe are seeking a highly skilled Senior Software Engineer to join our team in Bangalore, India. This is a unique opportunity to work on cutting-edge projects and contribute to the growth of our organization.As a Senior Software Engineer, you will be responsible for designing and developing web applications using the .NET framework, VB.NET, Win...
-
Senior Data Engineer
1 week ago
Bengaluru, Karnataka, India Kloud9 Full timeJob Description for Senior Data EngineerCompany OverviewKloud9 is a dynamic and innovative company that requires a skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will play a key role in designing, developing, and maintaining our data pipelines and ETL processes.ResponsibilitiesCollaborate with cross-functional teams to...
-
Senior Data Engineer
1 day ago
Bengaluru, Karnataka, India Mouri Tech (P) Ltd Full timeJob Title: Senior Data EngineerAbout Mouri Tech (P) LtdMouri Tech (P) Ltd is a leading technology company that requires experienced data engineers to drive our business growth. We are seeking a skilled Senior Data Engineer to join our team and help us develop innovative data-driven solutions.Salary: ₹1,200,000 - ₹2,500,000 per annumJob Description:We are...