Big Data Engineer with Batch Processing and API

3 days ago

Hyderabad, Telangana, India Caspex Corp Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Position: Big Data Engineer

Location: Hyderabad, India (Need to work from Experian- Hyderabad office at least 3 days in a week from day one)

No of Open positions: 1

Experience: 7+ Years

Job Description:

Job Responsibilities:

Analyzes, designs, develops, codes and implements programs and applications in one or more programming languages, for Big Data Applications. Supports applications with an understanding of system integration, test planning, scripting, and troubleshooting. Assesses the health and performance of software applications and databases. Establishes, participates, and maintains relationships with business units, customers and subject matter experts in order to remain apprised of direction, project status, architectural and technology trends, risks, and functional/integration issues. Analyzes program and application performance using various programming languages, tools and techniques. Proactively identify opportunities to optimize implementations for cost and execution efficiencies and show ownership and bias for action.

Required Qualifications:

  • Bachelors degree in Computer Science or related field or equivalent experience.
  • 5+ years of experience in analysis, coding, implementation and testing of Big Data Applications (API & Batch) on AWS Cloud using Java/Scala/Kotlin, Cassandra, , Python, Hadoop, HDFS, Spark, MapReduce framework Good AWS experience (Batch and API microservices) using EMR/EC2/ECS/S3/Airflow/Step Functions, AWS API Gateway Good Batch Application performance tuning (Spark/Hadoop/EMR, Java, python)

Preferred Qualifications:

  • Good Scala, Pyspark, Java coding experience for REST API and SOAP webservices Good Linux experience (Shell scripting/Python) Good to have DevOps experience (GIT, GitHub, Bit Bucket, GitLab, IntelliJ IDEA, PyCharm) Good to have experience creating CI/CD pipelines – Maven, Gradle, Jenkins, Artifactory, AWSCodeCommit / Cloudformation

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Sky Systems, Inc. (SkySys) Full time

    Role:Big Data EngineerPosition Type:Full-Time Contract (40hrs/week)Contract Duration:Long TermWork Schedule: 8 hours/day (Mon-Fri)Location: Hybrid - Hyderabad, India (3x days onsite/week)We are looking for a skilledBig Data Engineer (5+ years' experience)to join Experian's dynamic data engineering team. You will design and optimize scalable data pipelines,...

  • Big Data Engineer

    4 days ago

    Hyderabad, Telangana, India Sheryl strategic solutions Pvt. LTD . Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Description : Position : Big Data Engineer Location : Hyderabad, India (Hybrid, 2-3 times per week on any weekdays) Duration : 1 year Job Summary : Experian FSD is seeking a skilled and motivated Big Data Engineer to join our dynamic data engineering team. This hybrid role based in Hyderabad offers the opportunity to work on cutting-edge data...

  • Big Data Engineer

    1 week ago

    Hyderabad, Telangana, India Skilviu Soft Solutions Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Role: Big Data EngineerLocation: Hyderabad, India (Hybrid).Experience: 5+ YearsDuration: 1 year+Notice Period: Immediate -15 days.Key Responsibilities:Design, develop, and maintain scalable data processing pipelines using Spark/PySpark and Hadoop on AWS.Implement and optimize data workflows on AWS EMR, EC2, and ECS.Develop and integrate RESTful APIs and AWS...

  • Big Data Developer

    21 hours ago

    Hyderabad, Telangana, India MobMaxime Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Key Responsibilities:Design, develop, and maintain scalable big data systems and pipelines.Implement data processing frameworks and optimize large datasets using tools such as Hadoop, Spark, and Hive.Develop and maintain ETL processes to ensure data availability, accuracy, and quality for downstream applications.Collaborate with data architects, analysts,...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Techno Facts Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Experience in developing and delivering scalable big data pipelines using Apache Spark and Databricks on AWS.Position Requirements :Must Have : Build and maintain scalable data pipelines using Databricks and Apache Spark. Develop and optimize ETL/ELT processes for structured and unstructured data. Knowledge of Lakehouse architecture for efficient data...

  • AI Big Data Engineer

    6 days ago

    Hyderabad, Telangana, India Apple Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Apple is where individual imaginations gather together, committing to the values that lead to great work. Every new product we build, service we create, or Apple Store experience we deliver is the result of us making each other's ideas stronger. That happens because every one of us shares a belief that we can make something wonderful and share it with the...

  • AI Big Data Engineer

    1 week ago

    Hyderabad, Telangana, India Apple Full time ₹ 5,00,000 - ₹ 8,00,000 per year

    Apple is where individual imaginations gather together, committing to the values that lead to great work. Every new product we build, service we create, or Apple Store experience we deliver is the result of us making each other's ideas stronger. That happens because every one of us shares a belief that we can make something wonderful and share it with the...

  • LMTS - Big data Platform Engineer - Hyderabad

    2 weeks ago

    Hyderabad, Telangana, India griddable Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    DescriptionAbout SalesforceWe're Salesforce, the Customer Company, inspiring the future of business with AI+ Data +CRM. Leading with our core values, we help companies across every industry blaze new trails and connect with customers in a whole new way. And, we empower you to be a Trailblazer, too — driving your performance and career growth, charting new...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Incedo Inc Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Data Engineers – Big Data Platform At Incedo, we're looking for experienced Big Data Engineers (5–8 years) to join our AI & Data team in Hyderabad — and be part of building cutting-edge data ecosystems for the Banking domain. What You'll Do • Design, develop & optimize large-scale data pipelines using Hadoop, Hive, HDFS, and Kafka •...

  • Azure Data Engineer

    1 week ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...