Principal/Senior Data Engineer

2 days ago

Pune, India One Click AI Full time

Job Title : Principal / Senior Data Engineer Product Development

Location : Hyderabad / Pune (Onsite)

Job Type : Full-time, Payroll

Duration : Long-term

Job Description :

We are looking for an experienced Principal / Senior Data Engineer to join our product development team. You will design and build high-performance, distributed data systems that support large-scale data ingestion and processing. This role requires deep expertise in parallel processing, distributed computing, and modern cloud data platforms.

Key Responsibilities :

- Understand and translate complex design specifications into scalable solutions.

- Build and optimize data ingestion pipelines for distributed systems with parallel processing using Golang, C++, and Java.

- Ingest data from multiple sources including :

1. Cloud Storage : Amazon S3, Azure Blob, Google Cloud Storage

2. Relational Databases : Snowflake, Google BigQuery, PostgreSQL

3. Files, Kafka, Data Lakehouse (Iceberg)

- Implement high-availability (HA) loading, cross-region replication, and robust loading monitoring/error reporting.

- Develop and maintain Spark Connectors and integrate with third-party systems such as Kafka and Kafka Connect.

- Work in an Agile development environment and follow best practices for CI/CD pipelines.

Requirements :

- Strong experience building distributed systems with parallel processing.

- Hands-on expertise in one or more of the following : Kafka, Zookeeper, Spark, Stream Processing.

- Proficiency in Kafka Connect, Kafka Streams, Kafka security, and customization.

- Experience with Spark connectors and event-driven architectures.

- Familiarity with Agile methodologies and modern CI/CD practices.

Must-have Technical Skills :

- Programming : Java (4+ yrs), C++ (3+ yrs), Golang (3+ yrs) Advanced

- Data Platforms : Snowflake (3+ yrs Advanced), BigQuery (3+ yrs Intermediate), PostgreSQL (4+ yrs Advanced)

- Streaming & Processing : Apache Kafka (4+ yrs Advanced), Apache Spark (3+ yrs Intermediate)

- Cloud Storage : AWS S3 (4+ yrs Advanced), Azure (4+ yrs Advanced), Google Cloud (4+ yrs Intermediate)

- Distributed Computing : 2+ yrs experience

- Agile & CI/CD : 3- 4+ yrs Intermediate to Advanced

Nice to Have :

- Experience with gRPC protocol and multi-threading.

- Knowledge of Zookeeper / ETCD / Consul.

- Familiarity with distributed consensus algorithms (Paxos / Raft).

- Exposure to Docker and Kubernetes.


(ref:hirist.tech)

  • Principal data engineer

    4 days ago

    Pune, India Xebia Full time

    ???? We’re Hiring: Principal Data Engineer | Any Xebia Location (Hybrid, 3 days in office per week)???? Location: Any Xebia Location (Hybrid – 3 days in office per week)???? Experience Level: 10+ years of Data Engineering with 4+ years team leadership experience???? Start Date: Immediate to end of SeptemberAbout the Team:As a Principal Data Engineer in...

  • Senior Software Engineer

    2 weeks ago

    Pune, Maharashtra, India Principal Global Services Full time

    Responsibilities Indicative years of experience 5 yearsRole Description Principal Pune is hiring a Mainframe Modernization - Sr Infrastructure Engineer This engineer will be a part of the Platform support under Information Services IS and responsible for helping achieve the strategy around availability of our Mainframe environment through adoption of...

  • Senior/Principal Engineer

    2 weeks ago

    Pune, Maharashtra, India NorthStar HR Consultants Full time

    Job Title - Lead / Senior Principal Software Engineer - UI As a Principal Software Engineer (UI) , you will be instrumental in designing and developing intuitive, responsive, and scalable user interfaces that are critical to the success of our product. You will drive the strategic direction of UI architecture, championing best practices and ensuring our...

  • Principal Data scientist

    4 hours ago

    Pune, India YASH Technologies Full time

    We are seeking a highly experienced and visionary Senior Principal Data Scientist to join our dynamic team. This role is pivotal in driving our organization's advanced analytics initiatives and shaping our data-driven strategy. As a thought leader, you will lead complex data science projects, mentor junior scientists, and collaborate with cross-functional...

  • Senior Principal Software Development Engineer

    4 days ago

    Pune, India NorthStar HR Consultants Full time

    Job Description Position Overview As a Sr. Principal Software Development Engineer, you will be a key technical leader responsible for defining the long-term architecture and engineering strategy of our enterprise-grade cybersecurity platform. You will drive the design and evolution of highly scalable, reliable, and secure systems, influencing not just...

  • Senior Manager

    2 weeks ago

    Pune, Maharashtra, India Principal Financial Full time

    Responsibilities Senior Manager - Accounting This role will be part of Enterprise Financial Services EFS This role will lead the shared services functions Account payable Accounts Receivable General Ledger Reconciliation Expenses Allocation Business Support teams Tax and Sox Compliance The role will be integral part of Enterprise Finance team...

  • Software Senior Principal Engineer

    7 days ago

    Pune, Maharashtra, India Dell Technologies Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    The Software Engineering team delivers next-generation software application enhancements and new products for a changing world. Working at the cutting edge, we design and develop software for platforms, peripherals, applications and diagnostics — all with the most advanced technologies, tools, software engineering methodologies and the collaboration of...

  • Senior Data Analyst

    2 weeks ago

    Pune, Maharashtra, India Principal Global Services Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Responsibilities:Performing general analytics, statistical modeling on data sets in various areas of the business. Activities include data aggregation and insight development, analysis of data and presentation. Examines and identifies data patterns and trends to help answer business questions and improve decision making.A Senior Professional applies advanced...

  • Senior Data Analyst

    2 days ago

    Pune, India Principal Global Services Full time

    Responsibilities: Performing general analytics, statistical modeling on data sets in various areas of the business. Activities include data aggregation and insight development, analysis of data and presentation. Examines and identifies data patterns and trends to help answer business questions and improve decision making. A Senior Professional applies...

  • Senior/Principal Engineer

    5 days ago

    Pune, India NorthStar HR Consultants Full time

    Job Title - Lead / Senior Principal Software Engineer - UI As a Principal Software Engineer (UI) , you will be instrumental in designing and developing intuitive, responsive, and scalable user interfaces that are critical to the success of our product. You will drive the strategic direction of UI architecture, championing best practices and ensuring our...