Principal Data Engineer

3 weeks ago

mumbai, India Sony Pictures Networks India Full time
Principal Data Engineer:
Experience:
10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.
Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end Big Data solutions.
Advanced hands-on experience with:
ETL/ELT frameworks : (e.g., Apache Nifi, Airflow, dbt)
Data Warehousing : (e.g., Snowflake, Redshift, BigQuery)
Big Data Technologies : (e.g., Hadoop, Apache Spark, HDFS)
Streaming Data Platforms : (e.g., Apache Kafka, Kinesis, Flink)
Cloud Data Solutions : (AWS, GCP) with a focus on services like Lambda, EMR, Glue, and DataFlow.
Have 8+ year of experience in ClickHouse/Redshift or any other OLAP system
Proficient in data processing and programming with Python, Scala, or Java , leveraging these languages for building high-performance data pipelines.
Strong understanding of distributed computing , microservices architecture , and containerization technologies (e.g., Kubernetes, Docker).
Deep knowledge of data security, data governance, and compliance frameworks in large-scale, complex data ecosystems.
Experience in performance optimization and troubleshooting in high-throughput data environments.
worked in Hadoop eco-system with AWS cloud distribution and worked at least on one or more big data ingestion tools (Sqoop, Flume, NiFI, etc), distributed messaging and ingestion frameworks (Kafka, Pulsar, Pub/Sub, etc).
Core Roles and Responsibilities:
Architect and build large-scale, robust, and high-performance data pipelines that process both batch and streaming data.
Implement, automate, and maintain real-time data processing systems, ensuring high availability and low latency.
Collaborate with data scientists, machine learning engineers, and analysts to optimize data pipelines and integrate advanced analytics workflows.
Develop and maintain scalable ETL processes that handle vast amounts of structured and unstructured data from diverse sources.
Focus on continuous improvement, driving the evolution of architecture to ensure optimal performance, cost-efficiency, and scalability.
Optimize distributed systems for performance and cost-efficiency, including caching, compression techniques, and optimizing query performance.
Maintain, monitor, and troubleshoot complex distributed systems, implementing monitoring solutions and ensuring reliability through proactive alerting.
Lead technical deep dives into performance bottlenecks, debugging, and optimization of data processing workflows.
Manage and ensure data security, availability, and reliability by following best practices and aligning with industry standards (e.g., encryption, IAM policies).

  • Data Warehouse Engineer

    24 hours ago

    Mumbai, India Data Warehouse Engineer Full time

    Experience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.

  • Principal data engineer

    3 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    3 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    3 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: - 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. - Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    3 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Consultant

    4 weeks ago

    Mumbai, Maharashtra, India Plative Full time

    We are seeking a highly skilled Principal Data Consultant to join our team at Plative. As a Principal Data Consultant, you will be responsible for participating in complex and multidisciplinary data and business intelligence projects.Your primary focus will be on designing, implementing, and deploying data-centric solutions for clients across various...

  • Principal data engineer

    4 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...

  • Principal Data Engineer

    2 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...

  • Principal Data Engineer

    3 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...

  • Principal Data Engineer

    4 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...

  • Principal Data Engineer

    4 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    4 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    4 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    4 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    4 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Data Engineer and Principal Consultant Position

    1 day ago

    Mumbai, Maharashtra, India Genpact Full time

    About GenpactGenpact is a global professional services and solutions firm that delivers outcomes to shape the future. Our team of 125,000+ professionals across 30+ countries is driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients.Salary and BenefitsWe offer an estimated salary of $120,000 per annum based on...

  • Principal Software Engineer

    1 month ago

    Mumbai, Maharashtra, India Protegrity USA Full time

    Unlock Your Potential as a Principal Software Engineer at Protegrity USAAre you a skilled software engineer looking for a new challenge? Do you want to work with a talented team on cutting-edge projects? Look no further than Protegrity USA, a leading provider of data protection solutions.About the RoleWe are seeking a highly experienced Principal Software...

  • Principal BI Engineer

    3 months ago

    Mumbai, India Cornerstone OnDemand Full time

    We're looking for a Principal BI Engineer This role is Office Based, Pune Office Principal BI Engineer We are seeking a talented  Principal BI Engineer  in Pune, India. The right candidate has strong communication skills, passion for solving business problems with data, domain knowledge in Sales & Marketing, Customer Success & Product,...

  • Principal Engineer

    3 weeks ago

    Mumbai, Maharashtra, India Arcadis Full time

    At Arcadis, we are committed to attracting and developing top talent who share our passion for improving quality of life. As a Principal Engineer, you will have the opportunity to work on innovative projects and contribute to our mission of creating a more sustainable planet.Role DescriptionAs a Principal Engineer, you will lead design and delivery for...

  • Principal gcp data engineer

    3 weeks ago

    Mumbai, India IKS Health Full time

    What will you be responsible for? ● The ideal candidate will oversee end to end data pipeline stack, including design, development, testing, documenting and deploying the pipeline, by following the defined/proven data engineering practices in a cloud based environment (GCP). ● As principal you will provide subject matter expertise in Pipeline...