Thoucentric - Big Data Engineer - Hive/Hadoop

3 weeks ago

bangalore, India Thoucentric Full time

Job Description :


As a Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work with large datasets, perform data processing, and support various business functions by creating data pipelines, data processing jobs, and data integration solutions. You will be working in a dynamic and collaborative environment, leveraging your expertise in Hive, Hadoop, and PySpark to unlock valuable insights from our data.

Key Responsibilities :

Data Ingestion and Integration :


- Develop and maintain data ingestion processes to collect data from various sources.


- Integrate data from different platforms and databases into a unified data lake.


Data Processing :


- Create data processing jobs using Hive and PySpark for large-scale data transformation.



- Optimize data processing workflows to ensure efficiency and Pipeline Development:


- Design and implement ETL pipelines to move data from raw to processed formats.



- Monitor and troubleshoot data pipelines, ensuring data quality and Modeling and Optimization:


- Develop data models for efficient querying and reporting using Hive.



- Implement performance tuning and optimization strategies for Hadoop and Governance :


- Implement data security and access controls to protect sensitive information.



- Ensure compliance with data governance policies and best :


- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and provide data :


- Bachelor's degree in Computer Science, Information Technology, or a related field.


- 3+ years of experience in big data engineering and data processing.


- Proficiency in Hive, Hadoop, and PySpark.


- Strong SQL and NoSQL database experience.


- Experience with data warehousing and data modeling.


- Knowledge of data integration, ETL processes, and data quality.


- Strong problem-solving and troubleshooting skills.


- Excellent communication and teamwork skills.


Preferred Qualifications :


- Experience with cloud-based big data technologies (e.g., AWS EMR, Azure HDInsight, Google

Dataprep).


- Certification in Hadoop, Hive, or PySpark.


- Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes).


- Knowledge of data visualization tools (e.g., Tableau, Power BI).


(ref:hirist.tech)

  • Thoucentric - Big Data Engineer - Hive/Hadoop

    3 weeks ago

    Bangalore, India Thoucentric Full time

    Job Description : As a Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work with large datasets, perform data processing, and support various business functions by creating data pipelines, data processing jobs, and data integration solutions. You will be working in a dynamic and...

  • Thoucentric - Big Data Engineer - Hive/Hadoop

    4 weeks ago

    Bangalore, India Thoucentric Full time

    Job Description : As a Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work with large datasets, perform data processing, and support various business functions by creating data pipelines, data processing jobs, and data integration solutions. You will be working in a dynamic and...

  • Thoucentric - Big Data Engineer - Hive/Hadoop

    4 weeks ago

    Bangalore, Karnataka, India Thoucentric Full time

    Job Description :As a Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work with large datasets, perform data processing, and support various business functions by creating data pipelines, data processing jobs, and data integration solutions. You will be working in a dynamic and...

  • Thoucentric - Big Data Engineer - Hive/Hadoop

    3 weeks ago

    Bangalore, Karnataka, India Thoucentric Full time

    Job Description :As a Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work with large datasets, perform data processing, and support various business functions by creating data pipelines, data processing jobs, and data integration solutions. You will be working in a dynamic and...

  • Big Data Developer

    2 months ago

    bangalore, India Codersbrain technology pvt ltd Full time

    Salary : 12 - 20 LPA Requirements :- Bachelor's/Master's degree in Computer Science, Engineering, or a related field.- Master's degree preferred.- Proven experience as a Big Data Developer or similar role, with strong knowledge of Big Data and Hadoop ecosystems.- Hands-on experience with HDFS, Spark, MapReduce, Sqoop, and Hive.- Experience in loading and...

  • Big Data Developer

    3 weeks ago

    Bangalore, India Codersbrain technology pvt ltd Full time

    Salary : 12 - 20 LPA Requirements :- Bachelor's/Master's degree in Computer Science, Engineering, or a related field.- Master's degree preferred.- Proven experience as a Big Data Developer or similar role, with strong knowledge of Big Data and Hadoop ecosystems.- Hands-on experience with HDFS, Spark, MapReduce, Sqoop, and Hive.- Experience in...

  • Big Data Developer

    2 months ago

    Bangalore, Karnataka, India Codersbrain technology pvt ltd Full time

    Salary : 12 - 20 LPARequirements :- Bachelor's/Master's degree in Computer Science, Engineering, or a related field.- Master's degree preferred.- Proven experience as a Big Data Developer or similar role, with strong knowledge of Big Data and Hadoop ecosystems.- Hands-on experience with HDFS, Spark, MapReduce, Sqoop, and Hive.- Experience in...

  • Big Data Developer

    2 months ago

    Bangalore, India Codersbrain technology pvt ltd Full time

    Salary : 12 - 20 LPA Requirements :- Bachelor's/Master's degree in Computer Science, Engineering, or a related field.- Master's degree preferred.- Proven experience as a Big Data Developer or similar role, with strong knowledge of Big Data and Hadoop ecosystems.- Hands-on experience with HDFS, Spark, MapReduce, Sqoop, and Hive.- Experience in...

  • Big Data Engineer/Knowledge Graph Specialist

    1 month ago

    bangalore, India Thoucentric Full time

    Job Description - Design, build, and maintain scalable big data pipelines for processing and analyzing large datasets using Spark, Airflow, Bodo, Flume, Flink, etc.- Utilize technologies like Hadoop, Spark, Kafka, and NoSQL databases.- Develop data ingestion, transformation, and aggregation strategies.- Design and implement data warehouses and data marts...

  • Big Data Engineer/Knowledge Graph Specialist

    3 weeks ago

    bangalore, India Thoucentric Full time

    Job Description - Design, build, and maintain scalable big data pipelines for processing and analyzing large datasets using Spark, Airflow, Bodo, Flume, Flink, etc.- Utilize technologies like Hadoop, Spark, Kafka, and NoSQL databases.- Develop data ingestion, transformation, and aggregation strategies.- Design and implement data warehouses and data marts...

  • Thoucentric - Azure Data Engineer - DataLake/Data Factory

    3 weeks ago

    Bangalore, India Thoucentric Full time

    At Thoucentric, we work on various problem statements.The most popular ones are : Building capabilities that address a market need, basis our ongoing research efforts Solving a specific use case for a current or potential client based on challenges on-ground Developing new systems that help be a better employer and a better partner to clients All of these...

  • Thoucentric - Azure Data Engineer - DataLake/Data Factory

    2 months ago

    Bangalore, India Thoucentric Full time

    At Thoucentric, we work on various problem statements.The most popular ones are : Building capabilities that address a market need, basis our ongoing research efforts Solving a specific use case for a current or potential client based on challenges on-ground Developing new systems that help be a better employer and a better partner to clients All of these...

  • Thoucentric - Azure Data Engineer - DataLake/Data Factory

    2 months ago

    bangalore, India Thoucentric Full time

    At Thoucentric, we work on various problem statements.The most popular ones are : Building capabilities that address a market need, basis our ongoing research efforts Solving a specific use case for a current or potential client based on challenges on-ground Developing new systems that help be a better employer and a better partner to clients All of these...

  • Delta Lake Architect

    1 month ago

    bangalore, India Thoucentric Full time

    As a Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work with large datasets, perform data processing, and support various business functions by creating data pipelines, data processing jobs, and data integration solutions. You will be working in a dynamic and collaborative...

  • Delta Lake Architect

    3 weeks ago

    bangalore, India Thoucentric Full time

    As a Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work with large datasets, perform data processing, and support various business functions by creating data pipelines, data processing jobs, and data integration solutions. You will be working in a dynamic and collaborative...

  • Valiance Solutions

    2 weeks ago

    bangalore, India Valiance Analytics Pvt. Ltd. Full time

    About Us : Valiance is a global AI & Data analytics firm helping clients build cutting-edge technology solutions for digital transformation. We work with some of the marquee brands across India, US and APAC to build transformative solutions for Credit Risk, Fraud, Predictive Maintenance, Quality Inspection, Data lake, IOT analytics etc. Our team comprises...

  • Thoucentric - Technical Architect - ETL/Data Warehousing

    3 weeks ago

    Bangalore, India Thoucentric Full time

    At Thoucentric, we work on various problem statements. The most popular ones are : Building capabilities that address a market need, basis our ongoing research efforts Solving a specific use case for a current or potential client based on challenges on-ground Developing new systems that help be a better employer and a better partner to clients All of...

  • Thoucentric - Technical Architect - ETL/Data Warehousing

    2 months ago

    Bangalore, Karnataka, India Thoucentric Full time

    At Thoucentric, we work on various problem statements.The most popular ones are :Building capabilities that address a market need, basis our ongoing research effortsSolving a specific use case for a current or potential client based on challenges on-groundDeveloping new systems that help be a better employer and a better partner to clientsAll of these need...

  • Thoucentric - Technical Architect - ETL/Data Warehousing

    2 months ago

    bangalore, India Thoucentric Full time

    At Thoucentric, we work on various problem statements. The most popular ones are : Building capabilities that address a market need, basis our ongoing research efforts Solving a specific use case for a current or potential client based on challenges on-ground Developing new systems that help be a better employer and a better partner to clients All of...

  • Thoucentric - Technical Architect - ETL/Data Warehousing

    2 months ago

    Bangalore, India Thoucentric Full time

    At Thoucentric, we work on various problem statements. The most popular ones are : Building capabilities that address a market need, basis our ongoing research efforts Solving a specific use case for a current or potential client based on challenges on-ground Developing new systems that help be a better employer and a better partner to clients All of...