Big Data Engineer

4 days ago


Anywhere in IndiaMultiple Locations Changeleaders Full time

Key Responsibilities :

Data Pipeline Development :

- Design, develop, and maintain scalable data pipelines using Big Data technologies (e.g., Apache Hadoop, Apache Spark, Kafka).

- Implement data ingestion strategies to efficiently load and transform large datasets from various sources.

Data Architecture and Modeling :

- Collaborate with data architects to design and optimize data models that support analytical workloads.

- Create and maintain data schemas, ensuring efficient storage and retrieval of data across multiple platforms.

ETL Development :

- Develop ETL processes to extract, transform, and load data into data lakes and warehouses.

- Utilize tools like Apache NiFi, Talend, or custom-built solutions to automate data workflows.

Performance Tuning :

- Monitor and optimize the performance of data processing jobs to ensure high availability and reliability.

- Identify bottlenecks in data processing and implement solutions to improve efficiency.

Data Quality and Governance :

- Establish data quality frameworks to ensure accuracy, consistency, and reliability of data.

- Implement data governance practices, including metadata management and compliance with data regulations.

Collaboration and Stakeholder Engagement :

- Work closely with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical specifications.

Participate in cross-functional teams to develop data-driven solutions that support business objectives.

Troubleshooting and Support :

- Provide ongoing support for data systems, troubleshooting issues, and implementing fixes as necessary.

- Develop monitoring tools and dashboards to ensure data pipeline health and performance.

Documentation and Knowledge Sharing :

- Maintain thorough documentation of data processes, architectures, and standards for future reference.

- Share knowledge and best practices with team members through training sessions and collaborative discussions.

Continuous Improvement :

- Stay current with industry trends and emerging technologies in the Big Data landscape.

- Propose and implement enhancements to existing data solutions and practices.

Mentorship and Leadership :

- Mentor junior team members, providing guidance on technical challenges and career development.

- Lead initiatives to improve team processes and efficienc

- Document processes, data flows, and system architecture for future reference.

(ref:hirist.tech)

  • Anywhere in India/Multiple Locations US Technology International Pvt Ltd Full time

    Job Title: Senior Big Data EngineerJob Summary:US Technology International Pvt Ltd is seeking a Senior Big Data Engineer to lead the development and implementation of big data solutions. The ideal candidate will have expertise in Hadoop, Spark, and other big data technologies, as well as strong leadership and communication skills.Key Responsibilities:Design,...

  • Big Data Engineer

    2 weeks ago


    Anywhere in India/Multiple Locations D-TechWorks Pvt Ltd Full time

    We are looking for Aws big data engineer In Remote , India Position : Aws big data engineerExperience : 7 to 9 YearLocation : Remote , India Mandate skills : BigData Python Pyspark AWS Glue Airflow Redshift Responsibilities :- Ensure effective Design, Development, Validation and Support activities in line with client needs and architectural requirements.,-...

  • Big Data Engineer

    2 months ago


    Anywhere in India/Multiple Locations IT Source Global Full time

    Job SummaryWe are seeking a skilled Hadoop Developer to join our team at IT Source Global. As a Hadoop Developer, you will be responsible for designing and developing complex data pipelines for ingestion, transformation, and processing of structured and unstructured data.Key ResponsibilitiesDevelop, implement, and maintain large-scale data processing...

  • Big Data Architect

    3 days ago


    Anywhere in India/Multiple Locations Changeleaders Full time

    Job Summary: We are seeking an experienced Data Engineer to join our team at Changeleaders. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining scalable data pipelines using Big Data technologies.">About the Role: In this role, you will collaborate with cross-functional teams to develop data-driven...


  • Anywhere in India/Multiple Locations D-TechWorks Pvt Ltd Full time

    About the RoleWe are seeking an experienced Senior Big Data Engineer for Cloud Migration to join our team at D-TechWorks Pvt Ltd.Job SummaryThe ideal candidate will have 7-9 years of experience in leading big data engineering projects using AWS, Azure, or GCP. They should be proficient in BigData Python, PySpark, AWS Glue, Airflow, and Redshift.Key...

  • Lead Data Engineer

    1 week ago


    Anywhere in India/Multiple Locations Talentoj Full time

    Requirements : - 6+ years' experience with Big Data Technologies such as Apache Spark, Cloud Native, and Data Lakes, and Databricks. - Delivered ETL/ELT solutions including data extraction, transformation, cleansing, data integration, and data management. - Implemented batch ingestion pipelines. - Strong Python skills - automate tasks, write...


  • Anywhere in India/Multiple Locations Paralleldots Full time

    Job Description : As a Senior Data Engineer at ParallelDots, you will be responsible for overseeing the design and development of advanced data solutions. You will work closely with data scientists, analysts, and other stakeholders to ensure that our data infrastructure meets the highest standards of scalability, reliability, and : - Lead the design,...


  • Anywhere in India/Multiple Locations IT Source Global Full time

    About the RoleWe are seeking a skilled Big Data Engineer with Advanced Analytics Expertise to join our team at IT Source Global. This is an exciting opportunity for a talented individual to work on challenging projects, collaborating with data scientists and analysts to deliver innovative solutions.Key ResponsibilitiesDesign and develop complex data...

  • Big Data Engineer

    1 week ago


    Anywhere in India/Multiple Locations TalentXO Full time

    Required Qualifications : - 7-8 years of experience as a Data Engineer or in a similar role - Proficient in Python programming and PySpark for data processing and transformation - Extensive experience in designing and implementing data pipelines using AWS Glue and AWS MWAA (Airflow) - Strong SQL skills, including experience with AWS Redshift -...


  • Anywhere in India/Multiple Locations Paralleldots Full time

    About the Role:As a Senior Data Engineer at ParallelDots, you will play a pivotal role in designing and developing cutting-edge data solutions. Collaborating closely with data scientists, analysts, and stakeholders, you will ensure our data infrastructure meets the highest standards of scalability, reliability, and performance.Key Responsibilities:Lead the...


  • Anywhere in India/Multiple Locations IT Source Global Full time

    Job Description:As a Senior Big Data Architect at IT Source Global, you will play a key role in designing and developing large-scale data pipelines using cutting-edge technologies such as Py Spark, Python, and SQL. Your expertise will be instrumental in architecting scalable data lakes, data warehouses, and data marts that meet the evolving needs of our...


  • Anywhere in India/Multiple Locations US Technology International Pvt Ltd Full time

    About the Role We are seeking a highly skilled Senior Big Data Developer to join our team in Hyderabad. As a key member of our data engineering team, you will be responsible for designing, developing, and deploying high-performance big data solutions using Hadoop, Spark, and related technologies. About You To be successful in this role, you should have a...

  • Big Data Engineer

    2 months ago


    Anywhere in India/Multiple Locations/Remote Hum Technologies Full time

    Big Data Engineer with strong exp in Hadoop, Spark, SQL & AWS for one of the Fortune 500 Clients Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : : Apache NiFi, HDFS, Hive, Impala, Spark, Spark Streaming, Kafka, Java/Scala, SQL, Shell scripting Roles & Responsibilities : - Responsible for designing and implementing...


  • Anywhere in India/Multiple Locations US Technology International Pvt Ltd Full time

    Job Description : Responsibilities : 1. Explain and communicate the design / development to the customer 2. Perform and evaluate test results against product specifications 3. Break down complex problems into logical components 4. Develop user interfaces business software components 5. Use data models 6. Estimate time and effort required for...


  • Anywhere in India/Multiple Locations Aexonic Technologies Private Limited Full time

    Key Responsibilities :- Design, develop, and optimize Big Data pipelines and workflows using Hadoop, Hive, PySpark, and Google BigQuery.- Architect data solutions to facilitate smooth data migration and processing between on-premises systems and Google Cloud Platform (GCP).- Create and manage Unix Shell Scripts for process automation and data handling...


  • Anywhere in India/Multiple Locations Aexonic Technologies Private Limited Full time

    Key Responsibilities:- Design, develop, and optimize Big Data pipelines and workflows using Hadoop, Hive, PySpark, and Google BigQuery to ensure seamless data integration and processing.- Architect data solutions that facilitate smooth data migration and processing between on-premises systems and Google Cloud Platform (GCP) to drive business growth.- Create...

  • Data Engineer

    2 weeks ago


    Anywhere in India/Multiple Locations Orangeskill Technologies Pvt Ltd Full time

    Job Description :We are looking for a talented and experienced Data Engineer to join our team. You will be responsible for designing, building, and maintaining data pipelines, workflows, and infrastructure to support efficient data processing and analysis. The ideal candidate will have experience in workflow orchestration tools, database management, data...

  • Data Engineer

    2 weeks ago


    Metros/Anywhere in India/Multiple Locations One of the Consulting Firms Full time

    Data Engineer (Python & SQL)Position : Data Engineer (Python & SQL)Locations : Hyderabad, Chennai, Bengaluru, Mumbai, Pune, Kolkata, GurugramJob Type : Full-TimeCompany : Reputed Big 4 MNC client company of Live Connections.Role Overview :We are looking for a skilled Data Engineer with experience in Python, SQL, and Financial Services Industry (FSI) to join...

  • Cloud Data Engineer

    7 days ago


    Anywhere in India/Multiple Locations Excelra Knowledge Solutions Full time

    Job Description:Data engineers with expertise in ontologies, cloud platforms, and data technologies are sought after to build scalable data pipelines. The role involves designing, implementing, and managing cloud-based solutions on AWS and Azure.Key Responsibilities:Design and implement cloud infrastructure on AWS and AzureDevelop and optimize data pipelines...


  • Metros/Anywhere in India/Multiple Locations One of the Consulting Firms Full time

    Company OverviewWe are a leading consulting firm with extensive experience in providing data-driven solutions to our clients. Our team is comprised of highly skilled professionals who are dedicated to delivering exceptional results.Salary InformationThe estimated salary for this role is ₹12,00,000 per annum, which is competitive in the market and...