Big Data Developer

1 month ago


Kanpur, India Amossys Consulting Services Full time

Position Title: Data Engineer


Experience Required: 5+ years

Employment Type: Full-time

Employment Mode: Hybrid

Location: Bangalore, Pune, Trivandrum.

Industry: Banking & Insurance sector


Job Summary:

We are looking for a skilled Data Engineer with hands-on experience in PySpark and Big Data platforms to join our growing team. The ideal candidate will have strong expertise in building data pipelines, working with big data technologies like Apache Hive and Parquet, and handling diverse data use cases. You will work closely with data architects and analysts to ensure the accurate, efficient, and scalable processing of data.


Key Responsibilities:

  • Develop, optimize, and maintain scalable data pipelines using PySpark and other Big Data technologies.
  • Work with cloud-native services to ingest, process, and store large datasets.
  • Implement data processing solutions using Apache Hive, Parquet, and other big data storage formats.
  • Build end-to-end data pipelines for various use cases including reporting, system integration, and API endpoints.
  • Ensure data quality by applying data engineering best practices, testing, and validation methodologies.
  • Collaborate with stakeholders to understand data requirements and develop solutions for effective data management.
  • Analyse data using SQL to uncover insights and ensure efficient querying and reporting.
  • Maintain an understanding of data modelling principles and apply them to consumption scenarios.


Qualifications:

  • Bachelor’s degree in computers science, Engineering, or related field.
  • Min 5 years of experience in data engineering, with proven skills in PySpark and Big Data platforms.
  • Proficiency in Apache Hive, Parquet, and other Big Data technologies.
  • Experience with at least two end-to-end Big Data projects involving the ingestion, processing, and storage of data.
  • Knowledge of cloud-native services, with at least one project experience.
  • Solid data analysis skills using SQL.
  • Understanding of data modelling concepts.
  • Azure Synapse knowledge is desirable.


  • kanpur, India Dhira Full time

    Role: Data Engineer for AI/MLExperience Required: 6+ yearsWe’re looking for a true data craftsman who can dexterously navigate the worlds of raw data engineering and refined machine learning pipelines.You’re not the average “ETL engineer” who relies on pre-packaged tools like Informatica or Talend. You are a solid programmer, fluent in Python, and an...

  • Data Engineer

    2 months ago


    Kanpur, India Visa Full time

    Company DescriptionVisa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure...


  • Kanpur, India Lear Corporation Full time

    We’re Lear For YouLear, a global automotive technology leader in Seating and E-Systems, is Making every drive better by delivering intelligent in-vehicle experiences for customers around the world. With over 100 years of experience, Lear has earned a legacy of operational excellence while building its future on innovation. Our talented team is committed to...

  • Gen AI Architect

    6 days ago


    Kanpur, India IGT Solutions Full time

    Job Title: Architect - Gen AI, LLM, Big DataExperience: 18+ YearsLocation: Pune/ GurugramEmployment Type: Full-Time / PermanentJob Summary:We are looking for an experienced Architect with deep expertise in Generative AI (Gen AI), Large Language Models (LLM), and Big Data technologies. The ideal candidate will have 18+ years of experience in designing and...


  • kanpur, India Greenway Health Full time

    Minimum Qualifications• 5-7 years’ experience in Data Engineering – Data Processing using various tools• Excellent understanding and hands on experience with T-SQL programming• Excellent understanding and hands on experience with Microsoft SQL Server, other RDBMS, and No-SQL systems• Excellent understanding and hands on experience with ETL, Data...


  • Kanpur, Uttar Pradesh, India NTT DATA Full time

    Job DescriptionMake an impact with NTT DATAWe are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAThe Senior Associate Critical Facilities Operations and Maintenance...

  • Senior Engineer

    4 weeks ago


    Kanpur, Uttar Pradesh, India NTT DATA Full time

    Job DescriptionMake an impact with NTT DATANTT DATA is a $30+ billion trusted global innovator of business and technology services. We serve 75% of the Fortune Global 100 and are committed to helping clients innovate, optimize and transform for long-term success.Your day at NTT DATAThe Senior Associate Critical Facilities Operations and Maintenance Engineer...


  • Kanpur, Uttar Pradesh, India NTT DATA Full time

    About the RoleThe Senior Associate Critical Facilities Operations and Maintenance Engineer is a developing subject matter expert, responsible for working with all major systems within a facility or building such as electrical and HVAC systems and performs regular maintenance on all systems.This role helps to design, plan, install, and/or maintain mechanical,...


  • Kanpur, Uttar Pradesh, India NTT DATA Full time

    About the RoleJob DescriptionMake an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day...


  • Kanpur, Uttar Pradesh, India NTT DATA Full time

    About the RoleThe Senior Associate Specialist will play a key role in leading the delivery of multiple data centre construction projects of significant value.Key ResponsibilitiesLead a team of professionals to safely deliver major capital construction projects.Develop and maintain relationships with key customers, operations, finance, and internal department...


  • Kanpur, Uttar Pradesh, India Nippon Data Systems Ltd Full time

    ERP Business Development Manager Job SummaryWe are seeking a proactive ERP Business Development Manager to spearhead our growth in the ERP market. The successful candidate will be responsible for identifying new business opportunities, building strategic partnerships, and driving the expansion of our ERP solutions into new markets.Key...


  • Kanpur, Uttar Pradesh, India NTT DATA Full time

    Job DescriptionReq ID: 298087 We are seeking a skilled Technical Support Specialist to join our team in Bangalore, Uttar Pradesh (IN-UP), India (IN).As a Technical Support Specialist - Senior, you will be responsible for advising and assisting users in solving problems related to hardware, software, networks, and peripherals using available technology and...


  • Kanpur, India NTT DATA Full time

    Job Description Req ID: 299263 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Helpdesk Senior Associate - ITIL to join our team in Bangalore, Uttar Pradesh (IN-UP), India...


  • Kanpur, India NTT DATA Full time

    Job Description Req ID: 298154 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Helpdesk Senior Associate - ITIL to join our team in Bangalore, Uttar Pradesh (IN-UP), India (IN).Role...


  • Kanpur, India NTT DATA Full time

    Job Description Req ID: 298087 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Helpdesk Senior Associate - ITIL to join our team in Bangalore, Uttar Pradesh (IN-UP), India...


  • Kanpur, Uttar Pradesh, India NTT DATA Full time

    About the RoleWe are seeking a highly skilled Senior Associate Specialist to join our team at NTT DATA. As a key member of our construction team, you will be responsible for overseeing the delivery of major capital construction projects of significant value.Key ResponsibilitiesLead a team of professionals to safely deliver major capital construction projects...


  • Kanpur, Uttar Pradesh, India Recruin Full time

    About the RoleWe are seeking a highly skilled Quantitative Developer to join our team at Recruin. As a key member of our analytics group, you will play a critical role in developing and maintaining trading algorithms, analyzing market trends, and supporting data-driven decision-making.

  • Data Engineer

    3 weeks ago


    Kanpur, India MPACT TRAINING AND DEVELOPMENT Full time

    Data Engineer BangaloreExperience - 5 to 8 yrsJOB DESCRIPTION We are seeking experienced, motivated and enthusiastic Data Engineering Specialists to join our team. As part of the team, you will be involved in the implementation of the ongoing and new initiatives for our company and contribute to our mission of delivering high-quality data-driven solutions...

  • Data Analyst

    3 weeks ago


    Kanpur, Uttar Pradesh, India Brillius Technologies Full time

    Job Title: Junior Data AnalystBrillius Technologies is a leading provider of DevOps services and solutions, helping enterprises innovate and transform their application development and deployment processes.Key Responsibilities:Collect, analyze, and interpret data to drive business decisions and strategies.Apply data analysis, interpretation, and collection...

  • Senior Data Engineer

    2 weeks ago


    Kanpur, Uttar Pradesh, India Luxoft Full time

    Project OverviewWe are a leading software development company, Luxoft, hiring for project opportunities in the Middle East. Our team comprises front-end and back-end developers, data analysts and data scientists, architects, analysts, and project managers.You will be part of a fast-paced and exciting journey, working with modern technologies and being an...