Principal Data Engineer
3 weeks ago
Principal Data Engineer:
Experience:
- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.
- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end Big Data solutions.
- Advanced hands-on experience with:
- ETL/ELT frameworks: (e.g., Apache Nifi, Airflow, dbt)
- Data Warehousing: (e.g., Snowflake, Redshift, BigQuery)
- Big Data Technologies: (e.g., Hadoop, Apache Spark, HDFS)
- Streaming Data Platforms: (e.g., Apache Kafka, Kinesis, Flink)
- Cloud Data Solutions: (AWS, GCP) with a focus on services like Lambda, EMR, Glue, and DataFlow.
- Have 8+ year of experience in ClickHouse/Redshift or any other OLAP system
- Proficient in data processing and programming with Python, Scala, or Java, leveraging these languages for building high-performance data pipelines.
- Strong understanding of distributed computing, microservices architecture, and containerization technologies (e.g., Kubernetes, Docker).
- Deep knowledge of data security, data governance, and compliance frameworks in large-scale, complex data ecosystems.
- Experience in performance optimization and troubleshooting in high-throughput data environments.
- worked in Hadoop eco-system with AWS cloud distribution and worked at least on one or more big data ingestion tools (Sqoop, Flume, NiFI, etc), distributed messaging and ingestion frameworks (Kafka, Pulsar, Pub/Sub, etc).
Core Roles and Responsibilities:
- Architect and build large-scale, robust, and high-performance data pipelines that process both batch and streaming data.
- Implement, automate, and maintain real-time data processing systems, ensuring high availability and low latency.
- Collaborate with data scientists, machine learning engineers, and analysts to optimize data pipelines and integrate advanced analytics workflows.
- Develop and maintain scalable ETL processes that handle vast amounts of structured and unstructured data from diverse sources.
- Focus on continuous improvement, driving the evolution of architecture to ensure optimal performance, cost-efficiency, and scalability.
- Optimize distributed systems for performance and cost-efficiency, including caching, compression techniques, and optimizing query performance.
- Maintain, monitor, and troubleshoot complex distributed systems, implementing monitoring solutions and ensuring reliability through proactive alerting.
- Lead technical deep dives into performance bottlenecks, debugging, and optimization of data processing workflows.
- Manage and ensure data security, availability, and reliability by following best practices and aligning with industry standards (e.g., encryption, IAM policies).
-
Data Warehouse Engineer
12 hours ago
Mumbai, India Data Warehouse Engineer Full timeExperience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.
-
Principal data engineer
3 weeks ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Engineer
2 weeks ago
mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Engineer
3 weeks ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Consultant
4 weeks ago
Mumbai, Maharashtra, India Plative Full timeWe are seeking a highly skilled Principal Data Consultant to join our team at Plative. As a Principal Data Consultant, you will be responsible for participating in complex and multidisciplinary data and business intelligence projects.Your primary focus will be on designing, implementing, and deploying data-centric solutions for clients across various...
-
Principal data engineer
4 weeks ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...
-
Principal Data Engineer
3 weeks ago
mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...
-
Principal Data Engineer
3 weeks ago
mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Engineer
2 weeks ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...
-
Principal Data Engineer
3 weeks ago
mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...
-
Principal Data Engineer
4 weeks ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...
-
Principal Data Engineer
4 weeks ago
mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Engineer
4 weeks ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Engineer
4 weeks ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Engineer
4 weeks ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Engineer
4 weeks ago
mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Data Engineer and Principal Consultant Position
16 hours ago
Mumbai, Maharashtra, India Genpact Full timeAbout GenpactGenpact is a global professional services and solutions firm that delivers outcomes to shape the future. Our team of 125,000+ professionals across 30+ countries is driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients.Salary and BenefitsWe offer an estimated salary of $120,000 per annum based on...
-
Principal Software Engineer
1 month ago
Mumbai, Maharashtra, India Protegrity USA Full timeUnlock Your Potential as a Principal Software Engineer at Protegrity USAAre you a skilled software engineer looking for a new challenge? Do you want to work with a talented team on cutting-edge projects? Look no further than Protegrity USA, a leading provider of data protection solutions.About the RoleWe are seeking a highly experienced Principal Software...
-
Principal BI Engineer
3 months ago
Mumbai, India Cornerstone OnDemand Full timeWe're looking for a Principal BI Engineer This role is Office Based, Pune Office Principal BI Engineer We are seeking a talented Principal BI Engineer in Pune, India. The right candidate has strong communication skills, passion for solving business problems with data, domain knowledge in Sales & Marketing, Customer Success & Product,...
-
Principal Engineer
3 weeks ago
Mumbai, Maharashtra, India Arcadis Full timeAt Arcadis, we are committed to attracting and developing top talent who share our passion for improving quality of life. As a Principal Engineer, you will have the opportunity to work on innovative projects and contribute to our mission of creating a more sustainable planet.Role DescriptionAs a Principal Engineer, you will lead design and delivery for...
