Principal Data Engineer
1 month ago
Principal Data Engineer:
Experience:
- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.
- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end Big Data solutions.
- Advanced hands-on experience with:
- ETL/ELT frameworks: (e.g., Apache Nifi, Airflow, dbt)
- Data Warehousing: (e.g., Snowflake, Redshift, BigQuery)
- Big Data Technologies: (e.g., Hadoop, Apache Spark, HDFS)
- Streaming Data Platforms: (e.g., Apache Kafka, Kinesis, Flink)
- Cloud Data Solutions: (AWS, GCP) with a focus on services like Lambda, EMR, Glue, and DataFlow.
- Have 8+ year of experience in ClickHouse/Redshift or any other OLAP system
- Proficient in data processing and programming with Python, Scala, or Java, leveraging these languages for building high-performance data pipelines.
- Strong understanding of distributed computing, microservices architecture, and containerization technologies (e.g., Kubernetes, Docker).
- Deep knowledge of data security, data governance, and compliance frameworks in large-scale, complex data ecosystems.
- Experience in performance optimization and troubleshooting in high-throughput data environments.
- worked in Hadoop eco-system with AWS cloud distribution and worked at least on one or more big data ingestion tools (Sqoop, Flume, NiFI, etc), distributed messaging and ingestion frameworks (Kafka, Pulsar, Pub/Sub, etc).
Core Roles and Responsibilities:
- Architect and build large-scale, robust, and high-performance data pipelines that process both batch and streaming data.
- Implement, automate, and maintain real-time data processing systems, ensuring high availability and low latency.
- Collaborate with data scientists, machine learning engineers, and analysts to optimize data pipelines and integrate advanced analytics workflows.
- Develop and maintain scalable ETL processes that handle vast amounts of structured and unstructured data from diverse sources.
- Focus on continuous improvement, driving the evolution of architecture to ensure optimal performance, cost-efficiency, and scalability.
- Optimize distributed systems for performance and cost-efficiency, including caching, compression techniques, and optimizing query performance.
- Maintain, monitor, and troubleshoot complex distributed systems, implementing monitoring solutions and ensuring reliability through proactive alerting.
- Lead technical deep dives into performance bottlenecks, debugging, and optimization of data processing workflows.
- Manage and ensure data security, availability, and reliability by following best practices and aligning with industry standards (e.g., encryption, IAM policies).
-
Data Warehouse Engineer
1 week ago
Mumbai, India Data Warehouse Engineer Full timeExperience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.
-
Principal data engineer
4 weeks ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Engineer
4 weeks ago
mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Engineer
4 weeks ago
mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer: Experience: - 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. - Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Engineer
1 month ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal data engineer
1 month ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...
-
Principal Data Engineer
1 month ago
mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...
-
Principal Data Engineer
4 weeks ago
mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Engineer
3 weeks ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...
-
Principal Data Engineer
4 weeks ago
mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...
-
Principal Data Engineer
1 month ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...
-
Principal Data Engineer
1 month ago
mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Engineer
1 month ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Engineer
1 month ago
Mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Principal Data Engineer
1 month ago
mumbai, India Sony Pictures Networks India Full timePrincipal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...
-
Mumbai, Maharashtra, India Genpact Full timeAbout GenpactGenpact is a global professional services and solutions firm that delivers outcomes to shape the future. Our team of 125,000+ professionals across 30+ countries is driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients.Salary and BenefitsWe offer an estimated salary of $120,000 per annum based on...
-
Principal BI Engineer
3 months ago
Mumbai, India Cornerstone OnDemand Full timeWe're looking for a Principal BI Engineer This role is Office Based, Pune Office Principal BI Engineer We are seeking a talented Principal BI Engineer in Pune, India. The right candidate has strong communication skills, passion for solving business problems with data, domain knowledge in Sales & Marketing, Customer Success & Product,...
-
Principal gcp data engineer
4 weeks ago
Mumbai, India IKS Health Full timeWhat will you be responsible for? ● The ideal candidate will oversee end to end data pipeline stack, including design, development, testing, documenting and deploying the pipeline, by following the defined/proven data engineering practices in a cloud based environment (GCP). ● As principal you will provide subject matter expertise in Pipeline...
-
Principal GCP Data Engineer
4 weeks ago
mumbai, India IKS Health Full timeWhat will you be responsible for? ● The ideal candidate will oversee end to end data pipeline stack, including design, development, testing, documenting and deploying the pipeline, by following the defined/proven data engineering practices in a cloud based environment (GCP). ● As principal you will provide subject matter expertise in Pipeline...
-
Principal GCP Data Engineer
4 weeks ago
mumbai, India IKS Health Full timeWhat will you be responsible for?● The ideal candidate will oversee end to end data pipeline stack, including design, development,testing, documenting and deploying the pipeline, by following the defined/proven data engineeringpractices in a cloud based environment (GCP).● As principal you will provide subject matter expertise in Pipeline Architecture,...