Thoucentric - Big Data Engineer - Hive/Hadoop

3 weeks ago

Bangalore, Karnataka, India Thoucentric Full time

Job Description :


As a Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work with large datasets, perform data processing, and support various business functions by creating data pipelines, data processing jobs, and data integration solutions. You will be working in a dynamic and collaborative environment, leveraging your expertise in Hive, Hadoop, and PySpark to unlock valuable insights from our data.

Key Responsibilities :

Data Ingestion and Integration :


- Develop and maintain data ingestion processes to collect data from various sources.


- Integrate data from different platforms and databases into a unified data lake.


Data Processing :


- Create data processing jobs using Hive and PySpark for large-scale data transformation.



- Optimize data processing workflows to ensure efficiency and Pipeline Development:


- Design and implement ETL pipelines to move data from raw to processed formats.



- Monitor and troubleshoot data pipelines, ensuring data quality and Modeling and Optimization:


- Develop data models for efficient querying and reporting using Hive.



- Implement performance tuning and optimization strategies for Hadoop and Governance :


- Implement data security and access controls to protect sensitive information.



- Ensure compliance with data governance policies and best :


- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and provide data :


- Bachelor's degree in Computer Science, Information Technology, or a related field.


- 3+ years of experience in big data engineering and data processing.


- Proficiency in Hive, Hadoop, and PySpark.


- Strong SQL and NoSQL database experience.


- Experience with data warehousing and data modeling.


- Knowledge of data integration, ETL processes, and data quality.


- Strong problem-solving and troubleshooting skills.


- Excellent communication and teamwork skills.


Preferred Qualifications :


- Experience with cloud-based big data technologies (e.g., AWS EMR, Azure HDInsight, Google

Dataprep).


- Certification in Hadoop, Hive, or PySpark.


- Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes).


- Knowledge of data visualization tools (e.g., Tableau, Power BI).


(ref:hirist.tech)

  • Big Data Developer

    2 months ago

    Bangalore, Karnataka, India Codersbrain technology pvt ltd Full time

    Salary : 12 - 20 LPARequirements :- Bachelor's/Master's degree in Computer Science, Engineering, or a related field.- Master's degree preferred.- Proven experience as a Big Data Developer or similar role, with strong knowledge of Big Data and Hadoop ecosystems.- Hands-on experience with HDFS, Spark, MapReduce, Sqoop, and Hive.- Experience in...

  • Thoucentric - Technical Architect - ETL/Data Warehousing

    2 months ago

    Bangalore, Karnataka, India Thoucentric Full time

    At Thoucentric, we work on various problem statements.The most popular ones are :Building capabilities that address a market need, basis our ongoing research effortsSolving a specific use case for a current or potential client based on challenges on-groundDeveloping new systems that help be a better employer and a better partner to clientsAll of these need...

  • Unison Consulting

    2 months ago

    Bangalore, Karnataka, India Unison consulting pte ltd Full time

    We are seeking a talented Big Data Engineer to join our team and play a key role in designing, developing, and maintaining our big data infrastructure. You will leverage your expertise in Java, Hadoop, and Kafka to build efficient data pipelines, handle real-time data streams, and ensure high-quality data processing for our data analytics...

  • Unison Consulting

    3 weeks ago

    Bangalore, Karnataka, India Unison consulting pte ltd Full time

    We are seeking a talented Big Data Engineer to join our team and play a key role in designing, developing, and maintaining our big data infrastructure. You will leverage your expertise in Java, Hadoop, and Kafka to build efficient data pipelines, handle real-time data streams, and ensure high-quality data processing for our data analytics...

  • Thoucentric - Big Data Engineer/Knowledge Graph Specialist

    3 weeks ago

    Bangalore, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    Job Description :- Design, build, and maintain scalable big data pipelines for processing and analyzing large datasets using Spark, Airflow, Bodo, Flume, Flink, etc.- Utilize technologies like Hadoop, Spark, Kafka, and NoSQL databases.- Develop data ingestion, transformation, and aggregation strategies.- Design and implement data warehouses and data marts...

  • Thoucentric - Big Data Engineer/Knowledge Graph Specialist

    3 weeks ago

    Bangalore, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    Job Description :- Design, build, and maintain scalable big data pipelines for processing and analyzing large datasets using Spark, Airflow, Bodo, Flume, Flink, etc.- Utilize technologies like Hadoop, Spark, Kafka, and NoSQL databases.- Develop data ingestion, transformation, and aggregation strategies.- Design and implement data warehouses and data marts...

  • Thoucentric - Data Analyst

    1 month ago

    Bangalore, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    About us :Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally.As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project Management,...

  • Thoucentric - Data Analyst

    3 weeks ago

    Bangalore, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    About us :Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally.As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project Management,...

  • Thoucentric - Azure Data Engineer - Big Data Technologies

    3 weeks ago

    Bangalore, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    Job Description :Roles & Responsibilities :- Implement robust data pipeline using Microsoft Stack.- Create reusable and scalable data pipelines.- Development and deployment of new data platforms.- Leverage Azure BI services for development of Big Data Platforms.- Work closely with the Product Owners and Architects to develop Azure Data Platforms.- Work with...

  • Thoucentric - Azure Data Engineer - Big Data Technologies

    3 weeks ago

    Bangalore, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    Job Description :Roles & Responsibilities :- Implement robust data pipeline using Microsoft Stack.- Create reusable and scalable data pipelines.- Development and deployment of new data platforms.- Leverage Azure BI services for development of Big Data Platforms.- Work closely with the Product Owners and Architects to develop Azure Data Platforms.- Work with...

  • Thoucentric - Technical Consultant - ETL

    3 weeks ago

    Bangalore, Karnataka, India Thoucentric Full time

    About us : Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally.As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project...

  • Thoucentric - Technical Consultant - ETL

    1 month ago

    Bangalore, Karnataka, India Thoucentric Full time

    About us : Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally.As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project...

  • Big Data Engineer

    4 days ago

    Bangalore, Karnataka, India WEN Full time

    Role : Big Data Engineer (Scala)Responsibilities :- Develop and maintain scalable data pipelines using Spark and Hive within the Hadoop ecosystem- Utilize Scala programming language to implement efficient data processing solutions- Collaborate with cross-functional teams to understand data requirements and deliver optimal solutions- Implement best practices...

  • Data Engineer

    4 days ago

    Bangalore/Hyderabad, Karnataka, India Pan Asia Man power Consultants Recruiting agency Full time

    Data EngineerExperience Range : 6 to 14 YearsLocation : Bangalore (Hybrid: Work From Office - 3 Days in a Week)Full Time Skill Sets :- Experience designing and implementing data pipelines- Experience with SQL, RDBMS, or other data management, reporting and query tools.- Big Data Technologies - Hadoop, Hive, Spark, KafkaSecondary Skill Sets :- Added advantage...

  • Data Analyst

    3 weeks ago

    Bangalore, Karnataka, India Protiviti India Member Private limited Full time

    JOB DESCRIPTION :Roles and Responsibilities :- BTech/BE/BS/BA degree (ideally in a quantitative field such in Engineering, Statistics, Applied Econometrics, Math, Physics, Computer Science, other related Core Sciences or equivalent) with 4-5 years of experience in analytics and exposure to varied functions within analytics/finance OR - Master- s degree in...

  • Data Analyst

    3 weeks ago

    Bangalore, Karnataka, India Protiviti India Member Private limited Full time

    JOB DESCRIPTION :Roles and Responsibilities :- BTech/BE/BS/BA degree (ideally in a quantitative field such in Engineering, Statistics, Applied Econometrics, Math, Physics, Computer Science, other related Core Sciences or equivalent) with 4-5 years of experience in analytics and exposure to varied functions within analytics/finance OR - Master- s degree in...

  • Data Modeler

    1 month ago

    Bangalore, Karnataka, India Codersbrain technology pvt ltd Full time

    Job Description :The Data Modeler would perform data analysis of various systems and business context for defining requirements for data use cases across the Risk portfolios.- This will involve facilitating workshops, liaising with external and internal technical teams, initiative squads, business stakeholders, other data modelers and architecture/design...

  • Big Data Consultant

    2 months ago

    Bangalore, Karnataka, India Nilasu consulting Full time

    Summary :We are seeking a highly skilled consultant with expertise in Spark/Scala and Python to join our team. You will play a key role in developing and maintaining big data applications using Spark and Scala with a focus on functional programming principles. Additionally, your experience in Python and microservices architecture will be valuable in building...

  • Big Data Consultant

    3 weeks ago

    Bangalore, Karnataka, India Nilasu consulting Full time

    Summary :We are seeking a highly skilled consultant with expertise in Spark/Scala and Python to join our team. You will play a key role in developing and maintaining big data applications using Spark and Scala with a focus on functional programming principles. Additionally, your experience in Python and microservices architecture will be valuable in building...

  • Thoucentric - Site Reliability Engineer - Cloud Services

    3 weeks ago

    Bangalore, Karnataka, India Thoucentric Full time

    About us :Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally.As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project Management,...