Principal Data Engineer

4 weeks ago

mumbai, India Sony Pictures Networks India Full time

Principal Data Engineer:

Experience:

  • 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.
  • Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end Big Data solutions.
  • Advanced hands-on experience with:
  • ETL/ELT frameworks : (e.g., Apache Nifi, Airflow, dbt)
  • Data Warehousing : (e.g., Snowflake, Redshift, BigQuery)
  • Big Data Technologies : (e.g., Hadoop, Apache Spark, HDFS)
  • Streaming Data Platforms : (e.g., Apache Kafka, Kinesis, Flink)
  • Cloud Data Solutions : (AWS, GCP) with a focus on services like Lambda, EMR, Glue, and DataFlow.
  • Have 8+ year of experience in ClickHouse/Redshift or any other OLAP system
  • Proficient in data processing and programming with Python, Scala, or Java , leveraging these languages for building high-performance data pipelines.
  • Strong understanding of distributed computing , microservices architecture , and containerization technologies (e.g., Kubernetes, Docker).
  • Deep knowledge of data security, data governance, and compliance frameworks in large-scale, complex data ecosystems.
  • Experience in performance optimization and troubleshooting in high-throughput data environments.
  • worked in Hadoop eco-system with AWS cloud distribution and worked at least on one or more big data ingestion tools (Sqoop, Flume, NiFI, etc), distributed messaging and ingestion frameworks (Kafka, Pulsar, Pub/Sub, etc).

Core Roles and Responsibilities:

  • Architect and build large-scale, robust, and high-performance data pipelines that process both batch and streaming data.
  • Implement, automate, and maintain real-time data processing systems, ensuring high availability and low latency.
  • Collaborate with data scientists, machine learning engineers, and analysts to optimize data pipelines and integrate advanced analytics workflows.
  • Develop and maintain scalable ETL processes that handle vast amounts of structured and unstructured data from diverse sources.
  • Focus on continuous improvement, driving the evolution of architecture to ensure optimal performance, cost-efficiency, and scalability.
  • Optimize distributed systems for performance and cost-efficiency, including caching, compression techniques, and optimizing query performance.
  • Maintain, monitor, and troubleshoot complex distributed systems, implementing monitoring solutions and ensuring reliability through proactive alerting.
  • Lead technical deep dives into performance bottlenecks, debugging, and optimization of data processing workflows.
  • Manage and ensure data security, availability, and reliability by following best practices and aligning with industry standards (e.g., encryption, IAM policies).

  • Data Warehouse Engineer

    23 hours ago

    Mumbai, India Data Warehouse Engineer Full time

    Experience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.

  • Principal data engineer

    3 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    3 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    3 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: - 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. - Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    3 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Consultant

    4 weeks ago

    Mumbai, Maharashtra, India Plative Full time

    We are seeking a highly skilled Principal Data Consultant to join our team at Plative. As a Principal Data Consultant, you will be responsible for participating in complex and multidisciplinary data and business intelligence projects.Your primary focus will be on designing, implementing, and deploying data-centric solutions for clients across various...

  • Principal data engineer

    4 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...

  • Principal Data Engineer

    3 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    3 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...

  • Principal Data Engineer

    2 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...

  • Principal Data Engineer

    3 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...

  • Principal Data Engineer

    4 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...

  • Principal Data Engineer

    4 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    4 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    4 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    4 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Data Engineer and Principal Consultant Position

    1 day ago

    Mumbai, Maharashtra, India Genpact Full time

    About GenpactGenpact is a global professional services and solutions firm that delivers outcomes to shape the future. Our team of 125,000+ professionals across 30+ countries is driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients.Salary and BenefitsWe offer an estimated salary of $120,000 per annum based on...

  • Principal Software Engineer

    1 month ago

    Mumbai, Maharashtra, India Protegrity USA Full time

    Unlock Your Potential as a Principal Software Engineer at Protegrity USAAre you a skilled software engineer looking for a new challenge? Do you want to work with a talented team on cutting-edge projects? Look no further than Protegrity USA, a leading provider of data protection solutions.About the RoleWe are seeking a highly experienced Principal Software...

  • Principal BI Engineer

    3 months ago

    Mumbai, India Cornerstone OnDemand Full time

    We're looking for a Principal BI Engineer This role is Office Based, Pune Office Principal BI Engineer We are seeking a talented  Principal BI Engineer  in Pune, India. The right candidate has strong communication skills, passion for solving business problems with data, domain knowledge in Sales & Marketing, Customer Success & Product,...

  • Principal Engineer

    3 weeks ago

    Mumbai, Maharashtra, India Arcadis Full time

    At Arcadis, we are committed to attracting and developing top talent who share our passion for improving quality of life. As a Principal Engineer, you will have the opportunity to work on innovative projects and contribute to our mission of creating a more sustainable planet.Role DescriptionAs a Principal Engineer, you will lead design and delivery for...