Big Data Developer

2 months ago


Karnataka, India Amossys Consulting Services Full time

Position Title: Data Engineer


Experience Required: 5+ years

Employment Type: Full-time

Employment Mode: Hybrid

Location: Bangalore, Pune, Trivandrum.

Industry: Banking & Insurance sector


Job Summary:

We are looking for a skilled Data Engineer with hands-on experience in PySpark and Big Data platforms to join our growing team. The ideal candidate will have strong expertise in building data pipelines, working with big data technologies like Apache Hive and Parquet, and handling diverse data use cases. You will work closely with data architects and analysts to ensure the accurate, efficient, and scalable processing of data.


Key Responsibilities:

  • Develop, optimize, and maintain scalable data pipelines using PySpark and other Big Data technologies.
  • Work with cloud-native services to ingest, process, and store large datasets.
  • Implement data processing solutions using Apache Hive, Parquet, and other big data storage formats.
  • Build end-to-end data pipelines for various use cases including reporting, system integration, and API endpoints.
  • Ensure data quality by applying data engineering best practices, testing, and validation methodologies.
  • Collaborate with stakeholders to understand data requirements and develop solutions for effective data management.
  • Analyse data using SQL to uncover insights and ensure efficient querying and reporting.
  • Maintain an understanding of data modelling principles and apply them to consumption scenarios.


Qualifications:

  • Bachelor’s degree in computers science, Engineering, or related field.
  • Min 5 years of experience in data engineering, with proven skills in PySpark and Big Data platforms.
  • Proficiency in Apache Hive, Parquet, and other Big Data technologies.
  • Experience with at least two end-to-end Big Data projects involving the ingestion, processing, and storage of data.
  • Knowledge of cloud-native services, with at least one project experience.
  • Solid data analysis skills using SQL.
  • Understanding of data modelling concepts.
  • Azure Synapse knowledge is desirable.

  • Big Data Developer

    1 month ago


    Karnataka, India Amossys Consulting Services Full time

    Position Title: Data Engineer Experience Required: 5+ years Employment Type: Full-time Employment Mode: Hybrid Location: Bangalore, Pune, Trivandrum. Industry: Banking & Insurance sector Job Summary: We are looking for a skilled Data Engineer with hands-on experience in PySpark and Big Data platforms to join our growing team. The ideal candidate...

  • Big data developer

    1 month ago


    Karnataka, India Amossys Consulting Services Full time

    Position Title: Data Engineer Experience Required: 5+ years Employment Type: Full-time Employment Mode: Hybrid Location: Bangalore, Pune, Trivandrum. Industry: Banking & Insurance sector Job Summary: We are looking for a skilled Data Engineer with hands-on experience in Py Spark and Big Data platforms to join our growing team. The ideal...

  • Big Data Developer

    2 weeks ago


    Karnataka, India Advent Global Solutions Full time

    Position : Big Data Developer/Big Data Senior Developer Location : Chennai, Bangalore, Pune, Gurgaon, and MumbaiExperience : 5 to 14 yearsRelevant : should be 4+ yrs for Junior and 7+ yrs for SeniorPrimary skills : Spark, SQL,HiveDevelop and maintain architectures such as databases and large-scale processing systems using Python, Scala, SQL, and Unix...

  • Big Data Developer

    2 weeks ago


    Karnataka, India Advent Global Solutions Full time

    Position : Big Data Developer/Big Data Senior Developer Location : Chennai, Bangalore, Pune, Gurgaon, and Mumbai Experience : 5 to 14 years Relevant : should be 4+ yrs for Junior and 7+ yrs for Senior Primary skills : Spark, SQL,Hive Develop and maintain architectures such as databases and large-scale processing systems using Python, Scala, SQL, and Unix...

  • Big Data Architect

    6 months ago


    Bengaluru, Karnataka, India Unison Consulting Full time

    **Responsibilities**: Design and architect Big Data solutions to meet business requirements. Collaborate with stakeholders to understand their data and analytics needs. Create and maintain data models, data flow diagrams, and data integration strategies. Develop and implement data ingestion, processing, and storage solutions using Big Data...

  • Big Data

    6 months ago


    Bengaluru, Karnataka, India Quess IT Staffing Full time

    **About Us** “Magna Infotech, now Quess IT Staffing, is India’s largest IT staffing company with over 20 years of experience in staffing IT professionals in 300+ companies across levels and skillsets. Our 10,000+ associates deployed in 80+ cities and towns are proficient in over 500 technological skills. Our associates help enable cutting edge solutions...

  • Big Data

    2 months ago


    Bengaluru, Karnataka, India Virtusa Full time

    Skills from Primary Skill Cluster Big Data Pyspark Hadoop Skills outside Primary Skill Cluster **Job Description**: Responsibilities: We would be looking at Big Data + Spark professional. S/he should have excellent knowledge of SQL. Citi data lake is on Cloudera Hadoop with Hive. Other scripting knowledge like Python etc. are added advantage. Designs data...

  • Big Data Solutions

    5 months ago


    Bengaluru, Karnataka, India HCT INFOTECH Full time

    Big Data Solutions **About the Role** **Duration**:6 months **Location:Bangalore** **Timings: Notice Period**: (Immediate Joiner - Required)**Experience: 4-5 Years** **Must have skill sets**: - Strong development and coding experience in object-oriented programming languages. - Solid understanding of Micro-services Architecture and performance-oriented...

  • Data Engineer

    1 month ago


    Karnataka, India Tehno Right Full time

    Job Profile and Role:As a Data Engineer at Tehno Right, you will be responsible for designing and maintaining large-scale data pipelines, architectures, and data sets. You will work with Postgres and SQL, building and optimizing 'big data' data pipelines, and performing root cause analysis on internal and external data and processes to answer specific...


  • Karnataka, India NeerInfo Solutions Full time

    We are looking for a Data Architect / Sr. Data and Pr. Data Architects. This role involves a combination of hands-on contribution, customer engagement and technical team management. As a Data Architect, you will • Design, Architect, Deploy and maintain solutions on the MS Azure using different Cloud & Big Data Technologies. • Manage the full...


  • Karnataka, India NeerInfo Solutions Full time

    We are looking for a Data Architect / Sr. Data and Pr. Data Architects. This role involves a combination of hands-on contribution, customer engagement and technical team management. As a Data Architect, you will • Design, Architect, Deploy and maintain solutions on the MS Azure using different Cloud & Big Data Technologies. • Manage the full life-cycle...


  • karnataka, India NeerInfo Solutions Full time

    We are looking for a Data Architect / Sr. Data and Pr. Data Architects. This role involves a combination of hands-on contribution, customer engagement and technical team management. As a Data Architect, you will • Design, Architect, Deploy and maintain solutions on the MS Azure using different Cloud & Big Data Technologies. • Manage the full life-cycle...

  • Big Data and Gcp

    6 months ago


    Bengaluru, Karnataka, India Manage Captive Solutions Full time

    **Who We Are** Manage Captive Solutions (MCS) stands as the ultimate solution for effortless offshore hiring, precise payroll management, streamlined administration, and stringent compliances. We provide businesses distinctive solutions, making it easy to hire and onboard new team members. By removing the hurdles of setting up a local office, we enable...


  • karnataka, India Advent Global Solutions Full time

    Position : Big Data Developer/Big Data Senior Developer Location : Chennai, Bangalore, Pune, Gurgaon, and Mumbai Experience : 5 to 14 years Relevant : should be 4+ yrs for Junior and 7+ yrs for Senior Primary skills : Spark, SQL,Hive Develop and maintain architectures such as databases and large-scale processing systems using Python, Scala, SQL, and Unix...


  • karnataka, India Advent Global Solutions Full time

    Position : Big Data Developer/Big Data Senior Developer Location : Chennai, Bangalore, Pune, Gurgaon, and MumbaiExperience : 5 to 14 yearsRelevant : should be 4+ yrs for Junior and 7+ yrs for SeniorPrimary skills : Spark, SQL,HiveDevelop and maintain architectures such as databases and large-scale processing systems using Python, Scala, SQL, and Unix...

  • Big Data Tester

    5 months ago


    Bengaluru, Karnataka, India Talent On Technology Full time

    Reviews and refines software requirements. - Drafts and executes test plans and test cases. - Identifies and drafts steps to reproduce defects utilizing our defect tracking system, JIRA. - Enters and tracks defects to closure. - Utilizes testing tools to increase the effectiveness of testing. - Operate within the SAFe Agile methodology. Who we have in...

  • Big Data Architect

    6 months ago


    Bengaluru, Karnataka, India Virtusa Full time

    Position: Big data architect - Location: BLR - Primary skills: Big data hadoop spark & aws - Experience in building SparkStreaming process. Proficient in understanding distributed computing principles. Experience in managing Hadoop cluster, BigData with all services.Proficiency with Hadoop MapReduce HDFS Pig Hive and Impala. Experience with Nosql Databases...

  • Big Data Tester

    6 months ago


    Bengaluru, Karnataka, India Zuplon Full time

    **Big Data Tester (Manual + Automation)** **Preferred**: Prior Apple Experience - Advanced SQL - Required - python - Required - Full Stack-Java - Required - REST API Test Automation - Required - JUnit - Required - Unix Shell scripting - Nice To Have - Cloud Computing - Nice To Have - AWS- Nice To Have - Big Data Analytics - Nice To Have **Role...


  • Bengaluru, Karnataka, India EPAM Systems Full time

    RESPONSIBILITIES - Design data analytics solutions by utilising the big data technology stack - Create and present solution architecture documents with deep technical details - Work closely with business in identifying solution requirements and key case-studies/scenarios for future solutions - Conduct solution architecture review/audit, calculate and present...

  • Big Data Tester

    6 months ago


    Bengaluru, Karnataka, India Trident consulting Full time

    **Role**:SDET(Big Data) **Location**:Chennai/Bangalore/Hyderabad - Hybrid(3 days onsite/Week, Onsite from Day 1) **Type**:Fulltime **Skills**: - Strong SQL design and debugging skills - Big data Hadoop knowledge - Excellent hand on - Python automation - he should be able to automate his own test case and execute. **About Trident**: Trident Consulting is...