Data Pipeline Architect
7 days ago
**Key Responsibilities:**
As a senior data engineer on our team, you will be responsible for the following key responsibilities:
- Design, develop, and maintain scalable data pipelines in Azure Databricks using Python and Apache Spark for large-scale data processing.
- Collaborate with cross-functional teams to understand business needs and develop data solutions using Databricks.
- Use Python to develop custom data processing scripts, ETL jobs, and data transformation logic for handling structured, semi-structured, and unstructured data.
- Leverage Databricks notebooks for developing and managing workflows, data models, and analytical applications.
- Integrate data from cloud-based storage systems such as Azure Data Lake, Azure Blob Storage, and Azure SQL Database into Databricks.
- Optimize data pipelines for performance, scalability, and reliability, ensuring minimal latency and resource consumption.
- Implement and automate ETL processes to transform raw data into a structured, usable format for downstream consumption.
- Collaborate with data scientists and analysts to enable advanced analytics, machine learning, and reporting within Databricks.
- Ensure the security, compliance, and data governance of the Azure Databricks environment and integrate with existing Azure services.
- Troubleshoot, debug, and optimize data workflows, ensuring smooth data processing with minimal downtime.
- Document and maintain code, data models, and processes for reproducibility and transparency.
- Stay current with new features, trends, and best practices in Azure Databricks, Python, and cloud-based data technologies.
-
Cloud-Based Data Pipelines Architect
3 days ago
Mumbai, Maharashtra, India Pfizer Full timeWe are looking for a skilled Cloud-Based Data Pipelines Architect to join our global team at Pfizer. As a critical member of this team, you will be responsible for designing, developing, and implementing robust data pipelines that ingest and integrate data from various information sources.This role requires a strong understanding of modern data engineering...
-
Data Pipeline Architect
4 days ago
Mumbai, Maharashtra, India Sunday Labs Full timeAbout the Role:We are seeking a highly skilled Data Engineer with 6 to 10 years of experience at Sunday Labs in Mumbai (onsite). In this role, you will be responsible for designing, building, and maintaining large-scale data pipelines using SQL, ETL, and AWS technologies. You will collaborate closely with our Engineering Manager and Lead Data Engineer to...
-
Data Architect
2 weeks ago
Mumbai, Maharashtra, India Response Informatics Limited Full timeRoleData Architect. Skillset : PySpark, ADF and PowerBI. Experience10yrs to 18yrs. Location : ALL HCL Locations (3days work from office). Notice period: Immediate to 15days. Payroll Company: Response Informatics Ltd. Job Summary :- Experience as Data Architect with expertise in PySpark, Azure Data Factory (ADF), and Power BI to design and implement scalable...
-
Data Pipelines Specialist
2 days ago
Mumbai, Maharashtra, India Cadbury Full timeKey ResponsibilitiesDesign and build scalable, secure, and cost-effective cloud-based data solutionsManage data pipelines to extract, transform, and load data into data warehouses or data lakesEnsure data quality and validation processes to maintain data accuracy and integrityOptimize data storage and retrieval for optimal performanceCollaborate with data...
-
Senior Data Pipeline Specialist
22 hours ago
Mumbai, Maharashtra, India Thema Corporate Services Full timeAbout the RoleWe are seeking an experienced Senior Data Pipeline Specialist to join our team at Thema Corporate Services. In this role, you will be responsible for establishing connections to various data sources using Airbyte, configuring data extraction pipelines, and managing data replication schedules.You will also leverage dbt's declarative syntax for...
-
Data Pipeline Specialist
7 days ago
Mumbai, Maharashtra, India Servify Full timeJob SummaryWe are looking for a skilled Data Engineer to design, develop, and deploy efficient data pipelines using best-in-class technologies.The successful candidate will have experience with data life cycle management, advanced SQL concepts, and deployment using CI/CD and code management tools.They will also be responsible for managing structured and...
-
Cloud Data Architect
4 days ago
Mumbai, Maharashtra, India Go Digital Technology Consulting LLP Full timeJob DescriptionJob Description:At Go Digital Technology Consulting LLP (GDTC), we are redefining the future of data consulting and services. Our expertise spans Data Engineering, Analytics, and Data Science, enabling us to craft cutting-edge cloud data solutions tailored to our clients&apos unique needs. Specializing in AWS, Azure, and Snowflake, we empower...
-
Data Architect
2 weeks ago
Mumbai, Maharashtra, India Tata Consultancy Services Full timeRole - Azure Data ArchitectExperience - 7-13 yearsLocation - PAN IndiaJD :Responsibilities: Work with team of Architects and Designers to come up with end-to-end solutions to build data platforms on top of Azure, knowledge of Fabric platform preferred. Design and build scalable systems with Azure: use best practices and reducing Azure operational costs. Work...
-
Data Architect
3 weeks ago
Mumbai, Maharashtra, India Skill Ventory Full timeRoles and Responsibilities We are looking for skilled Azure Data Engineer / Architect to join our Data Lake team. The candidate shal have an experience into building and optimising large data platforms preferably in for lending business Exp required – - Yrs Skillsets Architecting Data platform solution Experience in delivery of large scale...
-
Senior Data Pipeline Developer
1 week ago
Mumbai, Maharashtra, India Citigroup Full timeCitigroup's Info Security Assessments team is seeking an experienced Database Specialist to design and implement scalable data pipelines and solutions. The ideal candidate will have a strong background in data engineering and a proven track record of delivering high-quality results.Main Responsibilities:Data Pipeline Development: Design and implement...
-
Data Architect
7 days ago
Mumbai, Maharashtra, India eClerx Investments Ltd Full timeJob OvervieweClerx Investments Ltd is seeking a skilled Data Architect to design and implement scalable, reliable, and maintainable data architectures on AWS.Key ResponsibilitiesDevelop data pipelines to extract, transform, and load (ETL) data from various sources into AWS environments.Create and optimize data models and schemas for performance and...
-
Data Architect
5 days ago
Mumbai, Maharashtra, India iamneo Full timeAbout the jobJob Summary:We are seeking a highly skilled Data Architect/Sr Data Engineer with experience on MS Azure or Cloudera platform. Hands on experience on Dataiku is must with at least 1 engagement and an overall 10+ years of experience in data engineering, ETL, and analytics solutions.The ideal candidate will be responsible for designing, developing,...
-
Data Architect
3 weeks ago
Mumbai, Maharashtra, India Innova ESI Full timeAbout the Role : - We are seeking a highly skilled and experienced Senior Data Architect to join our growing data engineering team.- As a Senior Data Architect, you will play a critical role in designing, developing, and implementing robust and scalable data solutions to support our business needs.- You will be responsible for defining data architectures,...
-
Data Architect
2 weeks ago
Mumbai, Maharashtra, India Innova ESI Full timeAbout the Role : - We are seeking a highly skilled and experienced Senior Data Architect to join our growing data engineering team.- As a Senior Data Architect, you will play a critical role in designing, developing, and implementing robust and scalable data solutions to support our business needs.- You will be responsible for defining data architectures,...
-
Big Data Pipeline Developer
7 days ago
Mumbai, Maharashtra, India Servify Full timeAbout ServifyServify is a leader in the service industry, providing innovative solutions to businesses across the globe. Our mission is to empower organizations to deliver exceptional customer experiences through cutting-edge technology and expertise.Job DescriptionWe are seeking an experienced Data Engineer to join our team. As a Data Engineer, you will be...
-
Mumbai, Maharashtra, India Akshya Patra Services Full timeAbout the job :Overview: The Talend + Snowflake Developer plays a crucial role in driving data integration and management initiatives within the organization.- This position requires a strong understanding of data warehousing concepts, data migration strategies, and ETL processes utilizing Talend to connect with Snowflake's cloud-based data platform.- As...
-
Mumbai, Maharashtra, India Akshya Patra Services Full timeAbout the job :Overview: The Talend + Snowflake Developer plays a crucial role in driving data integration and management initiatives within the organization.- This position requires a strong understanding of data warehousing concepts, data migration strategies, and ETL processes utilizing Talend to connect with Snowflake's cloud-based data platform.- As...
-
Zemoso Technologies Data Pipeline Specialist
18 hours ago
Mumbai, Maharashtra, India Zemoso Technologies Full timeJob Summary:Zemoso Technologies seeks a highly skilled Data Engineer to design and develop efficient data pipelines using Python.Main Accountabilities:Developing and maintaining data pipelines for smooth data flow and integration across systems.Setting up and maintaining Data Lakes, implementing industry best practices.Essential Qualifications:Bachelor's or...
-
Data Architect
2 weeks ago
Mumbai, Maharashtra, India Naabhik Solutions Full timeCompany DescriptionNaabhik Solutions is a data solution provider that helps enterprises create a reliable logical data architecture. By offering a toolset and data solution framework, Naabhik enables companies to understand and efficiently manage their data, gaining central governance over it. This allows enterprises to innovate, develop customer-centric...
-
Data Architect
2 weeks ago
Mumbai, Maharashtra, India Naabhik Solutions Full timeCompany DescriptionNaabhik Solutions is a data solution provider that helps enterprises create a reliable logical data architecture. By offering a toolset and data solution framework, Naabhik enables companies to understand and efficiently manage their data, gaining central governance over it. This allows enterprises to innovate, develop customer-centric...