Big Data Engineer with Batch Processing and API

5 days ago

Hyderabad, Telangana, India Caspex Corp Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Position: Big Data Engineer

Location: Hyderabad, India (Need to work from Experian- Hyderabad office at least 3 days in a week from day one)

No of Open positions: 1

Experience: 7+ Years

Job Description:

Job Responsibilities:

Analyzes, designs, develops, codes and implements programs and applications in one or more programming languages, for Big Data Applications. Supports applications with an understanding of system integration, test planning, scripting, and troubleshooting. Assesses the health and performance of software applications and databases. Establishes, participates, and maintains relationships with business units, customers and subject matter experts in order to remain apprised of direction, project status, architectural and technology trends, risks, and functional/integration issues. Analyzes program and application performance using various programming languages, tools and techniques. Proactively identify opportunities to optimize implementations for cost and execution efficiencies and show ownership and bias for action.

Required Qualifications:

  • Bachelors degree in Computer Science or related field or equivalent experience.
  • 5+ years of experience in analysis, coding, implementation and testing of Big Data Applications (API & Batch) on AWS Cloud using Java/Scala/Kotlin, Cassandra, , Python, Hadoop, HDFS, Spark, MapReduce framework Good AWS experience (Batch and API microservices) using EMR/EC2/ECS/S3/Airflow/Step Functions, AWS API Gateway Good Batch Application performance tuning (Spark/Hadoop/EMR, Java, python)

Preferred Qualifications:

  • Good Scala, Pyspark, Java coding experience for REST API and SOAP webservices Good Linux experience (Shell scripting/Python) Good to have DevOps experience (GIT, GitHub, Bit Bucket, GitLab, IntelliJ IDEA, PyCharm) Good to have experience creating CI/CD pipelines – Maven, Gradle, Jenkins, Artifactory, AWSCodeCommit / Cloudformation

  • Big Data engineer

    3 weeks ago

    Hyderabad, Telangana, India Cozzera Full time

    Job Title: Data Engineer Location: Hyderabad | Type: Full-time Description: We're hiring a Big Data Engineer to design and implement scalable data pipelines and streaming applications. You'll work with modern big data technologies and cloud platforms to power real-time data solutions. Key Skills: Strong software engineering with Java,...

  • 04/09/2025 Big Data engineer

    3 weeks ago

    Hyderabad, Telangana, India Cozzera Full time

    Job Title: Data Engineer Location: Hyderabad | Type: Full-time Description:We're hiring a Big Data Engineer to design and implement scalable data pipelines and streaming applications. You'll work with modern big data technologies and cloud platforms to power real-time data solutions. Key Skills:- Strong software engineering with Java, Scala, or Python-...

  • Big Data Developer

    1 day ago

    Hyderabad, Telangana, India KPG99 INC Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    About the Role:Experian FSD is seeking a skilled and motivated Big Data Engineer to join our dynamic data engineering team. This hybrid role based in Hyderabad offers the opportunity to work on cutting-edge data platforms and services that power Experian's global data ecosystem. You will be instrumental in building scalable data pipelines, integrating APIs,...

  • Big Data Engineer- Professional Services

    2 weeks ago

    Hyderabad, Telangana, India BrightEdge Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    About UsBrightEdge is a leading SEO and content performance marketing platform that transforms online content into tangible business results. Our platform processes massive amounts of data to provide actionable insights to our clients. We're looking for a talented Big Data Engineer to join our Professional Services to help us scale and optimize our data...

  • Big Data Engineer

    1 day ago

    Hyderabad, Telangana, India Techno-Comp Computer Services Pvt. Ltd. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    JOB RESPONSIBILITY· Experience with Big Data technologies will be a plus (Hadoop, Spark, Kafka, HBase, etc)·Write SQL queries to validate the dashboard output·Working experience with database environment - understanding relational database structure and hands-on SQL knowledge to extract/manipulate data for variance testing.·Performing code reviews and...

  • Brightedge - Big Data Engineer - ETL/Python

    3 weeks ago

    Hyderabad, Telangana, India brightedge Full time

    About UsBrightEdge is a leading SEO and content performance marketing platform that transforms online content into tangible business results.Our platform processes massive amounts of data to provide actionable insights to our clients.We're looking for a talented Big Data Engineer to join our Professional Services to help us scale and optimize our data...

  • Senior Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Warner Bros. Discovery Full time

    Job DescriptionRoles & Responsibilities- We are looking for a highly motivated Sr Data Engineer to build a state-of-the-art data platform to solve various data-driven use cases across the organization. This platform will host multiple data products, including but not limited to, Subscription, Content, and Product Analytics, Personalization and...

  • Senior Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Veltris Full time

    Position-Senior Big Data EngineerMust have Big Data analytics platform experience.• Key stacks: Spark, Druid, Drill, ClickHouse.• 8 years experience in Python/Java, CI/CD, infrastructure & cloud, Terraform, plus depth in:           o Big Data pipelines: Spark, Kafka, Glue, EMR, Hudi, Schema Registry, Data Lineage.           o Graph...

  • Big Data Infrastructure Specialist

    2 weeks ago

    Hyderabad, Telangana, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,10,00,000

    Job Opportunity:We are seeking a seasoned expert to spearhead the development and operation of our big data infrastructure projects. This role involves designing, building, and maintaining reliable and efficient data infrastructure that supports scalable data processing and analytics for both internal and external customers.Your Key Responsibilities:Develop...

  • Azure Data Engineer

    1 day ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...