Principal data engineer

1 month ago

Mumbai, India Sony Pictures Networks India Full time
Principal Data Engineer:
Experience:
10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.
Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end Big Data solutions.
Advanced hands-on experience with:
ETL/ELT frameworks : (e.g., Apache Nifi, Airflow, dbt)
Data Warehousing : (e.g., Snowflake, Redshift, Big Query)
Big Data Technologies : (e.g., Hadoop, Apache Spark, HDFS)
Streaming Data Platforms : (e.g., Apache Kafka, Kinesis, Flink)
Cloud Data Solutions : (AWS, GCP) with a focus on services like Lambda, EMR, Glue, and Data Flow.
Have 8+ year of experience in Click House/Redshift or any other OLAP system
Proficient in data processing and programming with Python, Scala, or Java , leveraging these languages for building high-performance data pipelines.
Strong understanding of distributed computing , microservices architecture , and containerization technologies (e.g., Kubernetes, Docker).
Deep knowledge of data security, data governance, and compliance frameworks in large-scale, complex data ecosystems.
Experience in performance optimization and troubleshooting in high-throughput data environments.
worked in Hadoop eco-system with AWS cloud distribution and worked at least on one or more big data ingestion tools (Sqoop, Flume, Ni FI, etc), distributed messaging and ingestion frameworks (Kafka, Pulsar, Pub/Sub, etc).
Core Roles and Responsibilities:
Architect and build large-scale, robust, and high-performance data pipelines that process both batch and streaming data.
Implement, automate, and maintain real-time data processing systems, ensuring high availability and low latency.
Collaborate with data scientists, machine learning engineers, and analysts to optimize data pipelines and integrate advanced analytics workflows.
Develop and maintain scalable ETL processes that handle vast amounts of structured and unstructured data from diverse sources.
Focus on continuous improvement, driving the evolution of architecture to ensure optimal performance, cost-efficiency, and scalability.
Optimize distributed systems for performance and cost-efficiency, including caching, compression techniques, and optimizing query performance.
Maintain, monitor, and troubleshoot complex distributed systems, implementing monitoring solutions and ensuring reliability through proactive alerting.
Lead technical deep dives into performance bottlenecks, debugging, and optimization of data processing workflows.
Manage and ensure data security, availability, and reliability by following best practices and aligning with industry standards (e.g., encryption, IAM policies).

  • Data Warehouse Engineer

    1 week ago

    Mumbai, India Data Warehouse Engineer Full time

    Experience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.

  • Principal data engineer

    4 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    4 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    4 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: - 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. - Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    1 month ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    1 month ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...

  • Principal Data Engineer

    4 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:- 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    3 weeks ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...

  • Principal Data Engineer

    4 weeks ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...

  • Principal Data Engineer

    1 month ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer:Experience:10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end...

  • Principal Data Engineer

    1 month ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    1 month ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    1 month ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    1 month ago

    Mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Principal Data Engineer

    1 month ago

    mumbai, India Sony Pictures Networks India Full time

    Principal Data Engineer: Experience: 10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments. Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of...

  • Data Engineer and Principal Consultant Position

    1 week ago

    Mumbai, Maharashtra, India Genpact Full time

    About GenpactGenpact is a global professional services and solutions firm that delivers outcomes to shape the future. Our team of 125,000+ professionals across 30+ countries is driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients.Salary and BenefitsWe offer an estimated salary of $120,000 per annum based on...

  • Principal BI Engineer

    3 months ago

    Mumbai, India Cornerstone OnDemand Full time

    We're looking for a Principal BI Engineer This role is Office Based, Pune Office Principal BI Engineer We are seeking a talented  Principal BI Engineer  in Pune, India. The right candidate has strong communication skills, passion for solving business problems with data, domain knowledge in Sales & Marketing, Customer Success & Product,...

  • Principal gcp data engineer

    4 weeks ago

    Mumbai, India IKS Health Full time

    What will you be responsible for? ● The ideal candidate will oversee end to end data pipeline stack, including design, development, testing, documenting and deploying the pipeline, by following the defined/proven data engineering practices in a cloud based environment (GCP). ● As principal you will provide subject matter expertise in Pipeline...

  • Principal GCP Data Engineer

    4 weeks ago

    mumbai, India IKS Health Full time

    What will you be responsible for? ● The ideal candidate will oversee end to end data pipeline stack, including design, development, testing, documenting and deploying the pipeline, by following the defined/proven data engineering practices in a cloud based environment (GCP). ● As principal you will provide subject matter expertise in Pipeline...

  • Principal GCP Data Engineer

    4 weeks ago

    mumbai, India IKS Health Full time

    What will you be responsible for?● The ideal candidate will oversee end to end data pipeline stack, including design, development,testing, documenting and deploying the pipeline, by following the defined/proven data engineeringpractices in a cloud based environment (GCP).● As principal you will provide subject matter expertise in Pipeline Architecture,...