Big data platform engineer

2 weeks ago


Bangalore, India Prodapt Full time

Roles and Responsibilities: Understand the requirements from Cloud user (Developers/Data scientists/Data analysts) and work with platform/infra teams to onboard their solutions in Big Query or Data Proc platforms. Create BQ datasets, Compute projects, Hive tables, GCS buckets and Service accounts etc. Grant access to BQ datasets, GCS buckets and Hive tables etc. Work closely with Leads, architects, product owners and PMO’s to enable new use cases in cloud. Administer Cloud platforms (Big Query and Data Proc) and manage reliability of platforms Monitor health of Cloud platforms, proactively identify the issues and address them Monitor the systems, Generate Performance Reports and provide continuous improvements. Develop frameworks to automate platform administration and monitoring tasks. Develop scripts to automate reports and maintenance activities. Work with cross functional teams to set up production environment and enable solutions for business. Debug day to day issues in cloud platforms and provide solutions. Perform Tuning and increase Operational efficiency on a continuous basis. Interact with business users to answer their queries and solve their problems. Able to help and Guide L1 support engineers to fix day-to-day Operational issues. Requirements: Minimum 6+ years of work experience and 4-5 years of work experience in Big Data technologies. Good experience in Level 2, 3 support for performing hot fixes and dealing with platform issues. Experience in working Google cloud platform or other cloud platforms will be plus. Good knowledge about Hadoop platform architecture, Hardware specs, Performance Benchmarking. Require good Unix skills and good knowledge about Networking architecture, Datacenter issues etc. Experience in tool Integration, automation, configuration management in GIT, Jira platforms. Should be proficient in writing shell scripts, automating the scripts and designing scheduler processes. Should be able to understand Python/Scala/Java programs and debug the issues. Excellent oral, written communication and presentation skills, analytical and problem-solving skills. Self-driven, Ability to work independently and as part of a team.



  • Bangalore, India Talpro Full time

    Job Description : Key Responsibilities :Design and Development :- Architect, design, and develop scalable and high-performance data platforms.- Implement data ingestion, transformation, and processing pipelines using technologies such as Apache Kafka, Hadoop, Spark, and Flink.- Develop real-time data processing and stream processing applications.Data...

  • Platform Architect

    2 months ago


    Bangalore, India Talpro Full time

    Educational Qualification : - BE./ B.Tech., M./M.Tech. or M.Sc. in Computer Science from Tier 1 university Skill Set :- 10+ years of experience in end-to-end software development and system architecture, preferably in a product development organisation- Hands-on experience in Java and Big Data tech stack - Apache Pinot/Druid/Hudi, HDFS, Hive, Presto, Flume,...


  • Bangalore, India Lakshya Software Technologies Pvt. Ltd. Full time

    Title: Cloud Big Data Engineer Exp - 5+ years Location: Anywhere in India Work Mode: HYBRID Mandatory Skills required: Big Data (Hadoop, Hive), Python, Spark, Py Spark, Cloud Platform, Azure Job description: Understanding of Big data Ecosystem with hands on experience working on Apache Spark, Hadoop (HDFS , Hive), Parquet, Elastic Search ...

  • Data Engineer

    4 weeks ago


    Bangalore, India Coders Brain Technology Private Limited Full time

    Qualifications : - Bachelor's degree in Computer Science/Engineering or similar - 5-7 years of data engineering experience with focus on designing and implementing scalable big data transformations and pipelines - Proficiency with Cloud Platforms like Azure(preferred), AWS, and/or Google cloud, including knowledge of core services like Azure: Data...

  • Solution Engineer

    4 weeks ago


    Bangalore, India HEVO DATA Full time

    Overview :. We're seeking a talented Solutions Engineer with strong background in Data Domain to join our growing Sales team. You'll work one-on-one with the sales team to help discover our customers' business and technical needs and demonstrate how Hevo's platform can solve their real-time data replication needs. In this role, you will...

  • Senior Data Engineer

    4 weeks ago


    Bangalore, India Visionyle Solutions Full time

    Title : Senior Data EngineerExperience : 6+ YearsNotice Period : Immediate to 15 Days (Serving NP candidates)Work Location - BangaloreMandatory skills : Big data applications, Spark, python, scala , AWS S3,Airflow,Data bricks, data platforms, Jenkins, CICD PipelinesRequirements :- 6+ years of experience in developing scalable Big Data applications or...


  • Bangalore, India FxConsulting Full time

    Job Title : Lead Data EngineerWe are seeking a highly skilled Lead Data Engineer with expertise in Big Data technologies to join our dynamic team. As a Lead Data Engineer, you will play a pivotal role in designing, building, and optimizing large-scale data pipelines and systems to drive business insights and innovation.Key Responsibilities : - Design,...


  • Bangalore, India Lakshya Software Technologies Pvt. Ltd. Full time

    Title: Cloud Big Data Engineer Exp - 5+ years Location: Anywhere in India Work Mode: HYBRID Mandatory Skills required: Big Data (Hadoop, Hive), Python, Spark, PySpark, Cloud Platform, Azure Job description: Understanding of Big data Ecosystem with hands on experience working on Apache Spark, Hadoop (HDFS , Hive), Parquet, Elastic Search ...

  • Big data engineer

    2 weeks ago


    Bangalore, India Capco Full time

    Job Title: Big Data Engineer About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount . With our presence across 32 cities across globe, we support 100+ clients...


  • Bangalore/Remote, India Skima Innovations Pvt ltd Full time

    About the Company : We are a stealth company revolutionizing the marketing landscape with our AI-powered platform. Our team has pioneered AI-based marketing systems at some of the most iconic companies in the world including Expedia, Amazon, Snapchat, and BCG to transform how businesses direct their marketing spend to drive unprecedented growth and...

  • Big data Engineer

    1 month ago


    Bengaluru / Bangalore, Gurgaon / Gurugram, India Talodyn Networks Private Limited Full time

    DescriptionWe are looking for an experienced Big Data Engineer to join our team. The ideal candidate will have 5-8 years of experience in the job market in India and will be responsible for designing, building, and maintaining our big data infrastructure. The candidate should have strong technical skills and experience working with big data technologies such...

  • Big data specialist

    2 weeks ago


    Bangalore, India LTIMindtree Full time

    Please fill this form for quick response:- Please fill this: - if it is not opening you can paste it on google and fill it. Experience Range - 5 to 12 yrs Skills - Big Data- Scala Spark Job Location: Bangalore, Chennai, Pune, Hyderabad, Mumbai Scala Spark Job Description: Mainly Scala+Spark ex perience needed. 5+ years of experience ...

  • Data Engineer

    4 weeks ago


    Mumbai/Hyderabad/Chennai/Pune/Bangalore, India Monster Full time

    Job Title: Data Engineer - Scalable Big Data SolutionsWe are seeking a seasoned Data Engineer with 5+ years of experience in designing and developing big data platforms using Scala and Spark. The ideal candidate will have expertise in data engineering, hands-on experience in designing and developing big data platforms, and a deep understanding of modern data...

  • Big Data Engineer

    2 weeks ago


    Bangalore, India Capco Full time

    Job Title: Big Data Engineer About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount . With our presence across 32 cities across globe, we support 100+...

  • Big Data Engineer

    1 month ago


    Bangalore, India Eximietas Design Full time

    6+ years of experience in Big data engineering with a focus on data pipeline setup and management. ● Strong development experience in Java ● Hands-on experience with Kafka , S3, CDC, Spark, and SQL. ● Proficient with container technologies like Docker. ● Experience working in cloud environments and handling large datasets. ● Solid...


  • Bangalore, India Draup Full time

    Role : Big Data EngineerJob Description :The Big Data Engineer at Draup is responsible for building scalable techniques and processes for data storage, transformations, and analysis. The role includes designing, and implementation of the optimal, generic, and reusable data-platforms and pipelines. You will work with a very proficient, inquisitive,...

  • Big Data Developer

    1 week ago


    Bangalore Urban, India Recro Full time

    Job Title : Big Data Engineer (Python & PySpark) Location : Bangalore Experience : 5+ years Key Responsibilities : Develop scalable data pipelines using Python and PySpark . Build and optimize Apache Spark jobs for data transformation and aggregation. Work with Big Data tools: Hadoop , HDFS , Hive , Kafka , and large datasets. Implement cloud-based...

  • Big Data Developer

    1 week ago


    Bangalore Urban, India Recro Full time

    Job Title: Big Data Engineer (Python & PySpark)Location: BangaloreExperience: 5+ yearsKey Responsibilities:Develop scalable data pipelines using Python and PySpark.Build and optimize Apache Spark jobs for data transformation and aggregation.Work with Big Data tools: Hadoop, HDFS, Hive, Kafka, and large datasets.Implement cloud-based solutions on Google Cloud...


  • bangalore, India Recro Full time

    Job Title: Data Engineer – Python, PySpark, Big DataLocation: [Onsite]Experience: [5+ years]Job Description:Seeking an experienced Data Engineer with expertise in Python, PySpark, and Big Data technologies to design and build scalable data solutions on the Google Cloud Platform (GCP).Responsibilities:Develop/optimize data pipelines using Python and...

  • Big Data Engineer

    4 weeks ago


    Bangalore, India Tavant Full time

    Key Responsibilities :- Design and implement scalable and efficient data processing pipelines using Apache Spark and Java.- Develop APIs and services for data ingestion, transformation, and aggregation to support analytics and machine learning models.- Collaborate with data scientists, analysts, and cross-functional teams to understand data requirements and...