Big Data Engineer

1 week ago


Bengaluru, Karnataka, India Tata Consultancy Services Full time
Job Description

We are seeking a skilled Big Data Engineer to design and implement ETL pipelines using PySpark for our Data Warehousing projects.

A successful candidate should have 3-6 years of experience in working with PySpark, Apache Spark, and Python programming.

  • Key Responsibilities:
  • Develop data processing tasks using PySpark, including reading data, merging, aggregating, enriching, and loading into target destinations.
  • Fine-tune processes and troubleshoot performance issues using Jupyter Notebook/PyCharm.
  • Experience with source code control systems like Git, GitHub, SVN, and deployment tools like Control-M.
  • CI/CD tools like Jenkins, Nexus, Ansible, and Agile delivery methodologies.
  • Unix/Linux Shell Scripting, data validation, unit test cases, and ETL Tools like Data Stage, Talend.
  • Databricks/Delta Lake experience is an added advantage.

Required Skills and Qualifications:

  • Bachelor's degree in Computer Science or related field.
  • Proficiency in Python programming and experience with PySpark.
  • Strong understanding of big data concepts, data warehousing, and ETL processes.
  • Excellent problem-solving skills and ability to work in a team environment.
  • Ability to learn new technologies and tools quickly.

Benefits:

  • Competitive salary: $120,000 - $180,000 per year.
  • Ongoing training and development opportunities.
  • Collaborative and dynamic work environment.
  • Opportunities for career growth and advancement.

  • Big Data Engineer

    4 weeks ago


    Bengaluru, Karnataka, India Krishpar Technologies Pvt. Ltd Full time

    Job Title: Big Data EngineerJob Summary: We are seeking a highly skilled Big Data Engineer to join our team at Krishpar Technologies Pvt. Ltd. The ideal candidate will have expertise in Hadoop, Hive, and other big data technologies. Key Responsibilities:Design and implement big data solutions using Hadoop, Hive, and other relevant technologies.Develop and...

  • Big Data Engineer

    1 month ago


    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Role: Big Data EngineerKey Responsibilities:Design and develop scalable big data architectures using Hadoop and SparkImplement data processing pipelines using Scala and SparkCollaborate with cross-functional teams to deliver data-driven insightsRequirements:5+ years of experience in big data engineeringStrong knowledge of Hadoop, Spark, Scala, and...

  • Big Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India TapTalent Full time

    We are seeking a talented Data Engineering Lead to join our team at TapTalent. As a key member of our data engineering team, you will be responsible for the design and development of our big data platform.Main Responsibilities:Expertise in Data Engineering with a focus on big data processing technologies.Deep understanding of modern data processing...


  • Bengaluru, Karnataka, India NTT DATA Services Full time

    About the RoleWe are seeking a highly skilled Cloud Data Engineer to join our team in Bangalore, India. As a key member of our data engineering team, you will design, build, and maintain scalable data pipelines using Databricks on AWS.Job Responsibilities:Design and develop efficient data pipelines using Apache Spark and Databricks.Ingest data from multiple...

  • Big Data Engineer

    1 month ago


    Bengaluru, Karnataka, India Hexaware Technologies Full time

    Job Title: Big Data DeveloperHexaware Technologies is seeking a highly skilled Big Data Developer to join our team. As a key member of our analytics team, you will be responsible for designing and developing distributed data systems, coding, testing, and documenting new or modified data systems to create robust and scalable applications for data...

  • Big Data Engineer

    1 month ago


    Bengaluru, Karnataka, India ATech Full time

    Job Title: Big Data EngineerWe are seeking a highly skilled Big Data Engineer to join our team at ATech. As a key member of our data engineering team, you will be responsible for designing, building, and optimizing our data infrastructure to support our growing data-driven organization.Key Responsibilities:Design and implement scalable data pipelines using...

  • Big Data Engineer 3

    3 weeks ago


    Bengaluru, Karnataka, India Oracle Full time

    Job SummaryWe are seeking an experienced Big Data Engineer to join our team at Oracle. This role involves designing, developing, and troubleshooting large-scale data systems.Key ResponsibilitiesDesign and implement data processing pipelines using Big Data technologiesDevelop and maintain large-scale data storage systemsCollaborate with cross-functional teams...

  • Big Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India Absolutdata Full time

    Location : Noida/BangaloreExperience : 4 – 8 YearsIn today's data-driven world, organizations rely on skilled professionals to unlock the power of big data. As a Data Engineer at Absolutdata, you will play a vital role in building the big data infrastructure, driving business growth, and delivering actionable insights.Key ResponsibilitiesDesign and...


  • Bengaluru, Karnataka, India Weekday Full time

    About the Role:We are seeking a highly skilled Big Data Solutions Engineer to join our team at Weekday. As a key member of our engineering team, you will be responsible for designing, developing, and implementing large-scale data processing systems using Big Data technologies such as Hadoop, Spark, and Elasticsearch.Responsibilities:Design and develop...


  • Bengaluru, Karnataka, India Grid Dynamics Full time

    Job Description:We are seeking a highly skilled Senior Big Data Engineer to join our team at Grid Dynamics. As a key member of our big data team, you will be responsible for designing, developing, and implementing large-scale data processing systems using Hadoop and Spring Boot.Responsibilities:- Design and develop scalable big data architectures using...


  • Bengaluru, Karnataka, India Nagarro Full time

    Job OverviewNagarro seeks a highly skilled Staff Engineer to join our team. The ideal candidate will have extensive experience in Big Data and a deep understanding of Hadoop and Data Lake.Key ResponsibilitiesLead the design and implementation of data-driven solutionsDevelop and deploy batch and streaming jobs on Hadoop and Data LakeCollaborate with...


  • Bengaluru, Karnataka, India Walmart Global Tech India Full time

    About the RoleWe are seeking an experienced Big Data Engineer to join our team at Walmart Global Tech India. As a key member of our engineering team, you will play a critical role in designing, building, and deploying cutting-edge big data solutions that drive business value for our customers.


  • Bengaluru, Karnataka, India TapTalent Full time

    Job Title: Senior Big Data Engineer - Cloud ExpertAt TapTalent, we are seeking a highly skilled Senior Big Data Engineer with expertise in Cloud Computing to join our team. As a Senior Big Data Engineer, you will be responsible for designing and implementing scalable data architectures using Cloud technologies. Your primary focus will be on building and...

  • AWS Big Data Engineer

    1 month ago


    Bengaluru, Karnataka, India ScaleneWorks Full time

    About the Role:We are seeking a highly skilled AWS Big Data Engineer to join our team at ScaleneWorks. The ideal candidate will have a strong background in coding and a deep understanding of big data technologies. You will be responsible for designing, developing, and maintaining big data solutions on AWS, ensuring data is processed efficiently and...

  • Big Data Engineer

    4 weeks ago


    Bengaluru, Karnataka, India Adobe Full time

    Transforming Digital Experiences with AdobeAt Adobe, we're committed to empowering creators and innovators to bring their ideas to life. As a Big Data Engineer, you'll play a critical role in shaping our data-driven future. What You'll DoDesign and develop cutting-edge data platforms and pipelines using advanced technologies, focusing on innovation and...

  • Data Engineer

    4 weeks ago


    Bengaluru, Karnataka, India Vyana Consultancy Full time

    Job RequirementsVyana Consultancy is seeking a highly skilled Data Engineer - Big Data Expert to join our team. The ideal candidate will have a strong background in big data technologies, including Hadoop, Spark, and NoSQL databases.Key ResponsibilitiesDesign, develop, and maintain large-scale data processing systemsCollaborate with cross-functional teams to...

  • Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India Mercedes-Benz Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Mercedes-Benz Research and Development India Private Limited. The ideal candidate will have deep expertise in Databricks, Python, and SQL queries, as well as experience in designing and implementing Big Data technologies.Key ResponsibilitiesDesign and implement Big Data...


  • Bengaluru, Karnataka, India GForce Consulting Solutions (OPC) Pvt. Ltd. Full time

    Senior Software EngineerGForce Consulting Solutions (OPC) Pvt. Ltd., a leading global IT services company, is seeking an experienced Senior Software Engineer for our Multinational Client.About the RoleThis role involves working with our client to design and develop cutting-edge big data solutions using AWS Data Bricks, S3, Redshift, EC2, Lambda, Azure data...

  • Big Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India Weekday Full time

    About the RoleThis position at Weekdays is designed to support the company's growth in data-driven solutions. The ideal candidate will possess expertise in programming languages such as Java, Python, or Scala, and have a strong background in Big Data technologies including Hadoop, MapReduce, Spark, HBase, and Elasticsearch.ResponsibilitiesWork as a skilled...

  • Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India ADCI - Karnataka Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our Music Data eXperience team at Amazon Music. As a Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data systems, including data pipelines, data warehouses, and data lakes.Key ResponsibilitiesDesign and develop data pipelines using big data...