Big Data Engineer

3 weeks ago

Pune, Maharashtra, India Merito Full time

Job Description :

We are looking for a Big Data Engineer who will work on building, and managing Big Data Pipelines for us to deal with the huge structured data sets that we use as an input to accurately generate analytics at scale for our valued Customers. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.

Core Responsibilities :

- Design, build, and maintain robust data pipelines (batch or streaming) that process and transform data from diverse sources.

- Ensure data quality, reliability, and availability across the pipeline lifecycle.

- Collaborate with product managers, architects, and engineering leads to define technical strategy.

- Participate in code reviews, testing, and deployment processes to maintain high standards.

- Own smaller components of the data platform or pipelines and take end-to- end responsibility.

- Continuously identify and resolve performance bottlenecks in data pipelines.

- Take initiatives, and show the drive to pick up new stuff proactively, and work as a Senior Individual contributor on the multiple products and features we have.

Required Qualifications :

- 5 to 7 years of experience in Big Data or data engineering roles.

- JVM based languages like Java or Scala are preferred.

- For someone having solid Big Data experience, Python would also be OK.

- Proven and demonstrated experience working with distributed Big Data tools and processing frameworks like Apache Spark or equivalent (for processing), Kafka or Flink (for streaming), and Airflow or equivalent (for orchestration).

- Familiarity with cloud platforms (e.g., AWS, GCP, or Azure), including services like S3, Glue, BigQuery, or EMR.

- Ability to write clean, efficient, and maintainable code.

- Good understanding of data structures, algorithms, and object-oriented programming.

Tooling & Ecosystem :

- Use of version control (e.g., Git) and CI/CD tools.

- Experience with data orchestration tools (Airflow, Dagster, etc.).

- Understanding of file formats like Parquet, Avro, ORC, and JSON.

- Basic exposure to containerization (Docker) or infrastructure-as-code (Terraform is a plus).

(ref:hirist.tech)

  • Big Data Engineer

    3 weeks ago

    Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full time

    Big Data Engineer (PySpark)Location: Pune/Nagpur (WFO)Experience: 8 - 12 YearsEmployment Type: Full-timeJob OverviewWe are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high...

  • Big Data Engineer

    4 weeks ago

    Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full time

    Big Data Engineer (PySpark) Location: Pune/Nagpur (WFO) Experience: 8 - 12 Years Employment Type: Full-time Job Overview We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high...

  • Big Data Engineer

    2 days ago

    Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Big Data Engineer (PySpark)Location:Pune/Nagpur (WFO)Experience:8 - 12 YearsEmployment Type:Full-timeJob OverviewWe are looking for an experiencedBig Data Engineerwith strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance...

  • Big Data Engineer

    5 days ago

    Pune, Maharashtra, India RiskInsight Consulting Pvt Ltd. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    ResponsibilitiesDesign, develop, and implement robust Big Data solutions using technologies such as Hadoop, Spark, and NoSQL databasesBuild and maintain scalable data pipelines for effective data ingestion, transformation, and analysisCollaborate with data scientists, analysts, and cross-functional teams to understand business requirements and translate them...

  • Big Data Engineer

    7 days ago

    Pune, Maharashtra, India Information Technology Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Big Data EngineerKey Responsibilities : - Build and manage highly scalable Big Data pipelines for large structured datasets. - Collaborate with Product Management & Engineering leadership to design and implement optimal solutions. - Participate in design discussions and contribute to selecting, integrating, and maintaining Big Data frameworks. -...

  • Big Data Engineer

    3 weeks ago

    Pune, Maharashtra, India WorkiFicient Full time

    To be successful in this role, you should possess : - Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.- Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data toolsand frameworks required to solve Big Data problems at scale.- Design and implement...

  • Big Data Engineer

    7 days ago

    Pune, Maharashtra, India Information Technology Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Key ResponsibilitiesBuild and manage highly scalable Big Data pipelines for large structured datasets.Collaborate with Product Management & Engineering leadership to design and implement optimal solutions.Participate in design discussions and contribute to selecting, integrating, and maintaining Big Data frameworks.Develop distributed data processing systems...

  • Big Data Developer

    2 days ago

    Pune, Maharashtra, India Wroots Global Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Title: Big Data Engineer / Software EngineerLocation: Pune Kharadi (Client Location) | WFO 3 days a weekExperience: 6 to 10 yearsJob Description:We are hiring an experienced Software Engineer with strong expertise in big data and cloud technologies.Key Skills:Hands-on experience in Scala / Java / PythonStrong knowledge of Big Data technologies: Spark,...

  • Big Data Engineer

    2 days ago

    Pune, Maharashtra, India Capco Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Title:Big Data Engineer - ScalaAbout Us"Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 byAvtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across...

  • Big Data Engineer

    3 weeks ago

    Pune, Maharashtra, India Tata Consultancy Services Full time

    Greetings from TCS TCS is hiring for Big Data Location: - Chennai/Mumbai/Pune Desired Experience Range: 6 to 12 years Must-Have • PySpark • Hive Good-to-Have • Spark • HBase • DQ tool • Agile Scrum experience • Exposure in data ingestion from disparate sources onto Big Data platform Thanks Anshika