Data Engineer

3 weeks ago


india SourceBae Full time

Job Title: Data Engineer

Exp-5+ years experience



Responsibilities:

  • Design, develop, and maintain scalable and efficient data pipelines and ETL processes using Python and PySpark.
  • Build and optimize data pipelines for large-scale data processing and analytics.
  • Implement data processing algorithms and data transformations for cleansing, aggregation, and analysis.
  • Collaborate with data scientists and analysts to understand data requirements and provide data engineering support.
  • Develop and maintain data warehouse solutions for storing and querying large volumes of structured and unstructured data.
  • Monitor and optimize the performance of data pipelines and processing jobs.
  • Troubleshoot and debug issues with data pipelines and data quality.
  • Ensure data security, privacy, and compliance with regulatory requirements.
  • Document data pipelines, processes, and infrastructure components.
  • Stay up-to-date with the latest technologies and trends in data engineering and big data processing.


Requirements:

  • 5+ years of experience working as a Data Engineer or similar role.
  • Strong programming skills in Python.
  • Hands-on experience with Apache Spark and PySpark.
  • Proficiency in SQL and experience with database technologies (e.g., PostgreSQL, MySQL, or similar).
  • Experience designing and building scalable data pipelines and ETL processes.
  • Knowledge of distributed computing concepts and frameworks.
  • Experience with cloud platforms such as AWS, Azure, or Google Cloud Platform.
  • Familiarity with containerization and orchestration tools like Docker and Kubernetes.
  • Experience with data warehousing solutions (e.g., Amazon Redshift, Google BigQuery, or similar).
  • Strong problem-solving and analytical skills.
  • Excellent communication and collaboration skills.
  • Bachelor's degree in Computer Science, Engineering, or a related field.


Nice to have:

  • Experience with streaming data processing frameworks (e.g., Apache Kafka, Apache Flink).
  • Knowledge of machine learning and data science concepts.
  • Experience with DevOps practices and tools.
  • Certification in relevant technologies (e.g., AWS Certified Big Data - Specialty, Databricks Certified Associate Developer for Apache Spark).



  • India Data Engineer Academy Full time

    Job Title: Operations Analyst Location : RemoteCompany Overview:Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary to...


  • india Data Engineer Academy Full time

    Job Title: Operations Analyst Location : Remote Company Overview: Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary...


  • india Data Engineer Academy Full time

    Job Title: Operations Analyst Location : Remote Company Overview: Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary...


  • india Lapis Data Analytics Full time

    Company Description Lapis Data Analytics is a dynamic and innovative data engineering company founded in 2020 and located in Chennai. Our primary focus is on designing, building, and maintaining robust data pipelines that convert raw data into actionable insights. With a team of seasoned experts and cutting-edge technology, we deliver customized data...


  • india K2 Data & Analytics Full time

    Job Description : We are on the lookout for an enthusiastic ML Engineer with over five years of experience to be part of our team, which is focused on developing cutting-edge Video Ad Data technologies. The perfect fit for this role would be someone with a solid grasp of machine learning algorithms and a background in designing and executing large-scale...

  • Aptus Data Labs

    2 weeks ago


    india Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • MLOps Engineer

    2 weeks ago


    india GOOGNU DATA SERVICES PRIVATE LIMITED Full time

    Job Description :- Design and Implement MLOps Infrastructure: Build and maintain robust MLOps workflows using Azure Machine Learning and other Azure services to automate machine learning lifecycle from data preparation to model deployment and monitoring.- Collaborate with Cross-Functional Teams: Work closely with data scientists, software developers, and IT...

  • QA Engineer

    1 week ago


    india Data Unveil Full time

    Position Title: QA Engineer Level: Mid-Level Location: Hyderabad, Telangana. Hire Type: Full Time Start Date: Immediate Minimum 3+ years of QA experience in data projects. Collaborate with the Development/Business teams to understand the requirements. Analyzing Business Requirements/cases and developing/managing the positive/negative test scenarios cases....

  • Data Engineer

    2 weeks ago


    india JRD Systems Full time

    Role : Data Engineer - AWS and Oracle Expert Job Summary: We are seeking a highly skilled and experienced Data Engineer with expertise in data development in AWS and Oracle. The ideal candidate will have a minimum of 5 years of experience in data engineering, with a strong focus on AWS and Oracle technologies. The Data Engineer will be responsible for...


  • India Liberty Data Analytics Full time

    Senior Specialist, Data Solutions Provides, builds, and maintains data solutions to meet the needs of the enterprise. Helps ensure those solutions can be scaled appropriately. Under minimal to no direction, works within broad limits and authority on highly complex assignments requiring specialized knowledge. Viewed as senior resource in given field for...


  • india Liberty Data Analytics Full time

    Senior Specialist, Data Solutions Provides, builds, and maintains data solutions to meet the needs of the enterprise. Helps ensure those solutions can be scaled appropriately. Under minimal to no direction, works within broad limits and authority on highly complex assignments requiring specialized knowledge. Viewed as senior resource in given field for...

  • Data Engineer

    4 weeks ago


    india HTC Global Services Full time

    Position Description: • This role is for experienced Data Engineer that will be responsible for designing and building the foundational components required for our customer. This includes bringing disparate sources into our Data Factory and creating data products that will be used by the analytics and reporting teams. • Data engineers will work in small,...


  • india NTT DATA Full time

    Job Description NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future. In today’s ‘iNTTerconnected’ world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help...

  • Data Engineer

    1 month ago


    india Joveo Full time

    We are looking for talented Data Engineers who are passionate about data engineering and have strong experience in building scalable data pipelines. As a Data Engineer at Joveo, you will play a leading role in designing and building datasets that power analytics for teams across Joveo. You will also be working closely with Data Analysts, Data Scientists, and...

  • Data Engineer

    1 month ago


    India Straive Full time

    Job Role- AWS Data EngineerExperience- 4 to 6 yearsMode- RemoteThe Data Engineering team is looking for a Data Engineer with experience in building modern data platforms from the ground up. The successful candidate will build and maintain cloud-centric data processing capabilities that unleash the value of the data assets to gain competitive advantage in the...

  • Data Engineer

    1 month ago


    india Straive Full time

    Job Role- AWS Data Engineer Experience- 4 to 6 years Mode- Remote The Data Engineering team is looking for a Data Engineer with experience in building modern data platforms from the ground up. The successful candidate will build and maintain cloud-centric data processing capabilities that unleash the value of the data assets to gain competitive advantage in...

  • Data Engineer

    1 month ago


    india ValueLabs Full time

    Experience - 6+Years Primary Skills - ADF, SQL Server, Databricks NoteBooks, Data Pipelines, SPROCs, Data Warehouse, Databricks DataLake Secondary Skills - PowerBl, Data modeling Notice Period - Immediate - 15 days Job Description: The Data Engineer is responsible for designing, developing, and maintaining data pipelines and data warehouses to support...

  • Data Engineer

    5 hours ago


    india Evive | goevive.com Full time

    We’re looking for a thorough Data Engineer. Is it you? The Data Engineer designs and builds the data pipelines that will enable faster and more accurate data-informed recommendations and decisions both inside and outside of the Evive platform. On a daily basis they design and build services to collect, compile, normalize, and analyze data assets coming...

  • Data Engineer

    5 hours ago


    india WaferWire Cloud Technologies Full time

    Role: Data Engineer Experience: 3–6 Years Location: Hyderabad Job Description We are seeking a seasoned Data Engineer to join our team in Hyderabad. The ideal candidate will have a strong background in data engineering, with extensive experience in SQL, data pipelines, Azure Synapses Data Warehouses, and data modeling. Key Responsibilities: Design,...

  • Data Engineer

    3 weeks ago


    india NAB Full time

    Consultant - Data Engineer role exists to ensure the delivery of the Right data to the Right customer at the Right time and at the Right Cost using the Right tools and the Right Technology on NAB’s target state Marketing Platforms. You will enable the move, flow and processing of reusable data assets in the Technology and Enterprise Operations space for...