Big Data Infrastructure Specialist

4 weeks ago


Bengaluru, Karnataka, India PeopleX Ventures Full time

Job Description:

We are looking for a skilled Data Platform Developer to join our team at PeopleX Ventures. In this role, you will be responsible for designing, building, and managing our cloud-based data infrastructure. You will develop scalable, reliable, and efficient data processing systems that support strategic business decisions.

About the Role:

The successful candidate will have a strong background in data engineering and cloud computing. They will be responsible for building and maintaining robust data pipelines to ingest, process, and transform large volumes of data (100s of terabytes daily). They will optimize data pipelines for performance and cost-efficiency, ensuring data quality and integrity through rigorous validation and monitoring processes.

Key Responsibilities:

  • Design and implement distributed data processing systems using Apache Spark, Kafka, and other big data technologies.
  • Develop data models and schemas to support complex analytical queries.
  • Collaborate with data scientists and analysts to understand their needs and provide efficient data solutions.
  • Manage and optimize cloud-based data infrastructure (AWS, GCP).
  • Implement monitoring and alerting systems to proactively identify and resolve issues.
  • Ensure high availability and disaster recovery for critical data systems.

Required Skills and Experience:

  • Strong proficiency in Python and PySpark.
  • Deep understanding of Apache Spark, including RDDs, DataFrames, and performance optimization techniques.
  • Experience with big data technologies such as Hadoop, HDFS, YARN, MapReduce, Hive, Kafka, and Airflow.
  • Familiarity with NoSQL databases and distributed database systems.
  • Experience with cloud platforms (AWS, GCP) and cloud-native services.
  • Strong problem-solving and analytical skills.
  • Excellent communication and collaboration skills.
  • 5+ years of experience in data engineering or software engineering roles.

Benefits:

  • Competitive salary: $150,000 - $200,000 per year.
  • Ongoing training and professional development opportunities.
  • A collaborative and dynamic work environment.


  • Bengaluru, Karnataka, India o9 Solutions, Inc. Full time

    We are seeking a highly skilled Big Data Infrastructure Specialist to join our team at o9 Solutions, Inc.As a key member of our infrastructure team, you will be responsible for designing, implementing, and maintaining large-scale data processing systems using Spark, Delta Lake, and Hadoop.The ideal candidate will have 8+ years of experience in DevOps, with a...


  • Bengaluru, Karnataka, India TMRW Full time

    Estimated Salary: $120,000 - $180,000 per yearThe ideal candidate will have at least 2+ years of experience in data engineering and data infrastructure space on big data technologies such as Hive, Spark, Pyspark(Batch and Streaming), Airflow, Redshift, and Delta Lake.


  • Bengaluru, Karnataka, India Data Inc Full time

    Job OverviewData Inc seeks a seasoned Chief Data Infrastructure Specialist to join our Data & Analytics Office (DAO). The ideal candidate will possess extensive experience in designing and implementing robust data management policies, ensuring data quality, regulatory compliance, and trusted sources.Key Responsibilities


  • Bengaluru, Karnataka, India Nielsen Full time

    Job DescriptionWe are seeking an experienced Senior Big Data Specialist to join our team at Nielsen. As a Senior Big Data Specialist, you will play a key role in analyzing, developing, testing, and supporting big data applications.Key ResponsibilitiesDesign, code, and test iteratively to support complex big data applications deployed on cloud infrastructure...


  • Bengaluru, Karnataka, India PeopleX Ventures Full time

    About the Opportunity :We are seeking an experienced Big Data Infrastructure Developer to join our team at PeopleX Ventures. As a key member of our engineering team, you will be responsible for designing, building, and managing our cloud-based data infrastructure.Key Responsibilities :Develop scalable, reliable, and efficient data processing systems.Design...


  • Bengaluru, Karnataka, India MathCo Full time

    Job SummaryWe are seeking a skilled Senior Data Infrastructure Specialist to join our team at MathCo. In this role, you will be responsible for designing, developing, and maintaining large-scale data infrastructure solutions that meet the evolving needs of our business.About MathCoMathCo is a leading organization in the industry, committed to delivering...


  • Bengaluru, Karnataka, India big basket Full time

    Job DescriptionAs a Product Analytics Specialist at Big Basket, you will be responsible for analyzing complex data to inform business decisions and drive growth. This is an exciting opportunity to join a fast-paced, data-driven environment and make a meaningful impact on the business.Data Analysis & InsightsAnalyze large datasets to identify emerging...


  • Bengaluru, Karnataka, India Sail Analytics Full time

    **Job Summary:** Big Data Engineering SpecialistWe are looking for a highly skilled Big Data Engineering Specialist to join our team at Sail Analytics. As a big data engineer, you will be responsible for designing, developing, testing, and maintaining scalable data warehouses and data pipelines using a range of technologies including SQL, Python, and...


  • Bengaluru, Karnataka, India LinkedIn Full time

    Designing and Building InfrastructureAs a Cloud Infrastructure Specialist, you will design and build scalable infrastructure and backend services for our data processing pipelines. You will work with our cross-functional teams to drive alignment on product and technology strategy, and lead cross-team discussions to ensure successful delivery of projects....


  • Bengaluru, Karnataka, India Blue Silicon Infotech Private Limited Full time

    About Blue Silicon Infotech Private LimitedWe are a cutting-edge technology firm committed to delivering innovative solutions that drive business growth. Our team of experts is dedicated to providing exceptional services that meet the evolving needs of our clients.Job OverviewThis role offers an exciting opportunity to join our IT infrastructure team as an...


  • Bengaluru, Karnataka, India Visionyle Solutions Full time

    Visionyle Solutions is hiring a talented Big Data Applications Specialist to join our team. As a key member of our data engineering team, you will be responsible for designing and developing scalable Big Data applications or solutions on distributed platforms.The estimated salary for this role is ₹1,200,000 - ₹2,000,000 per annum, depending on...


  • Bengaluru, Karnataka, India Luxoft Full time

    Job DescriptionLuxoft is seeking a highly skilled Big Data Analytics Specialist to join our team. In this role, you will be responsible for managing analytics support for key initiatives across Retail Bank supporting countries and group teams.Responsibilities:Ensuring high-quality service output, submitted in a timely manner and communicated...


  • Bengaluru, Karnataka, India LinkedIn Full time

    As a Cloud Infrastructure Architect - Big Data at LinkedIn, you will design and build infrastructure and backend services at internet scale. With a strong background in building large-scale distributed systems, applications, or similar experience, and over 14 years of experience in Java, C++, or similar programming languages, you will be a key member of our...


  • Bengaluru, Karnataka, India EduRun Full time

    EduRun is a leading education technology company that is seeking a skilled Big Data Processing Specialist. The successful candidate will have experience with Java programming, big data frameworks, and data analysis tools.The key responsibilities of this role include designing, developing, and maintaining high-performance Java applications, utilizing big data...


  • Bengaluru, Karnataka, India Synechron Full time

    Job DescriptionWe are seeking a highly skilled Big Data Management Specialist to join our team at Synechron.This is a fantastic opportunity for an experienced professional to work with cutting-edge technologies and contribute to the success of our organization.The ideal candidate will have extensive knowledge of Hadoop, HQL, Hive, Oozie, and Informatica Big...


  • Bengaluru, Karnataka, India Grid Dynamics Full time

    Job DescriptionWe are seeking an experienced Big Data Engineering Specialist to join our team at Grid Dynamics in Bangalore.The ideal candidate should have a minimum of 6-8 years of experience in software development, specifically in building large-scale applications using open-source technologies like Hadoop, Spark, Hive, and MapReduce.You will be...


  • Bengaluru, Karnataka, India Synechron Full time

    About the RoleWe are seeking a skilled Big Data Testing Specialist to join our team. As a key member of our data quality assurance team, you will be responsible for ensuring the integrity, reliability, and performance of our big data solutions.Key Responsibilities:Develop comprehensive test plans and cases for ETL processes and big data solutions.Execute...


  • Bengaluru, Karnataka, India Synechron Full time

    We are seeking a skilled professional to join our team as a Big Data Quality Assurance Specialist. This role is ideal for someone with a strong background in big data and ETL testing.About the RoleAs a Big Data Quality Assurance Specialist, you will be responsible for ensuring the integrity, reliability, and performance of our data solutions. Your primary...


  • Bengaluru, Karnataka, India Infometry Full time

    Company Overview:Infometry is a leading provider of innovative data solutions. We are committed to delivering high-quality services to our clients and believe in fostering a collaborative and dynamic work environment. Our company culture values innovation, creativity, and teamwork, making us an attractive workplace for professionals looking to grow their...


  • Bengaluru, Karnataka, India EXIMIETAS DESIGN PRIVATE LIMITED Full time

    About Eximietas Design Private LimitedWe are a leading design firm seeking a highly skilled Senior Big Data Specialist to join our team.The ideal candidate will have expertise in big data engineering, specifically focused on designing, building, and managing large-scale data systems. Strong development experience with Java and familiarity with scripting...