Data Engineer

2 weeks ago

India Stealth AI Startup Full time

The Data Engineer will be responsible for developing real-time and batch data integration pipelines. This role will ensure that data pipelines adhere to defined standards and are included in the established data integration framework. The Data Engineer will work closely with data scientists, analysts, and other stakeholders to ensure data availability for analysis and decision-making, significantly impacting our data-driven initiatives

Key Role Responsibilities:

  • Data Pipeline Development: Create and maintain scalable and efficient data pipelines for data extraction, transformation, and loading (ETL) processes, ensuring data accuracy and consistency.
  • Data Warehousing: Collaborate with Sr. Data Engineers and Data Architects to design and manage data lakes, storing and organizing large volumes of structured and unstructured data.
  • Data Modeling: Develop data models and schema designs that optimize data storage and retrieval, supporting various analytical and reporting needs.
  • Data Integration: Integrate data from multiple sources, including APIs and external databases, for real-time and batch data pipelines.
  • Data Quality Assurance: Implement data quality checks, validation, and cleansing processes to maintain data integrity.
  • Documentation: Create and maintain documentation for data engineering processes, data schemas, and ETL workflows.
  • Performance Optimization: Monitor and fine-tune data pipelines to ensure optimal performance and efficiency.
  • Collaboration: Work with cross-functional teams, including data scientists, analysts, and software engineers, to understand data requirements and deliver solutions.

Qualifications, Experience, Skills & Competencies:

Minimum Qualifications:

  • Graduate or Post-graduate in Computer Science or a related field is preferred.

Minimum Experience:

  • Minimum of 3 years of IT experience with at least 2 years in a similar role.

Job-Specific Skills:

  • Knowledge of big data technologies such as Hadoop, Spark, Kafka, and related ecosystems.
  • Proficient in SQL and experience with data warehousing platforms like Snowflake, Redshift, or BigQuery.
  • Strong experience with ETL/ELT tools such as Alteryx, Talend, or Informatica.
  • Proficiency in programming languages, particularly Python, Java, or Scala.
  • Experience with database technologies, including SQL and NoSQL databases.
  • Familiarity with cloud platforms and services such as AWS, Azure, or Google Cloud.
  • Experience with data orchestration tools like Apache Airflow or Informatica.

Behavioral Competencies:

  • Practical understanding of agile methodologies.
  • Ability to work in small teams structured around delivering data products.
  • Proven problem-solving skills for unstructured, uncertain problems.
  • Data-oriented, analytical, performance-driven, and assertive.

Technical Competencies:

  • ETL Tools (Informatica, Google DataFlow, Data Factory): Expert
  • Real-Time (Apache Kafka, Google Pub-Sub): Expert
  • Data Modeling / Data Integration: Advanced
  • GBQ or Synapse or Oracle Autonomous DWH: Advanced
  • PowerBI / Alteryx: Basic
  • Python / Jupyter Notebooks: Advanced

  • Operations Specialist

    2 weeks ago

    India Data Engineer Academy Full time

    Job Title: Operations Analyst Location : Remote Company Overview: Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary...

  • Data Architect

    2 weeks ago

    India Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • Aptus Data Labs

    6 days ago

    india Aptus Data Labs Full time

    Description : We are seeking a highly skilled and experienced individual to join our team as a Data Governance expert with AWS knowledge. In this role, you will play a critical role in ensuring data integrity, quality, and security across our organization's data assets. As a data governance expert, your main responsibility will be to develop and implement...

  • Aptus Data Labs

    4 days ago

    india Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • MLOps Engineer

    4 days ago

    india GOOGNU DATA SERVICES PRIVATE LIMITED Full time

    Job Description :- Design and Implement MLOps Infrastructure: Build and maintain robust MLOps workflows using Azure Machine Learning and other Azure services to automate machine learning lifecycle from data preparation to model deployment and monitoring.- Collaborate with Cross-Functional Teams: Work closely with data scientists, software developers, and IT...

  • Lead Data Engineer

    4 days ago

    india Advance Data Analytics Ltd. Full time

    *Immediate Joiners Preferred* We are hiring for a UK based client Experience: 6 to 10 years Location: Noida (Hybrid) Your main duties & responsibilities: Design, build and unit test of aspects data services, from ingestion, through harmonization and transformation into business usable analytics. Input into High level design and responsibility for low...

  • QA Engineer

    2 months ago

    india Data Unveil Full time

    Position Title: QA Engineer Level: Mid-Level Location: Hyderabad, Telangana. Hire Type: Full Time Start Date: Immediate Minimum 3+ years of QA experience in data projects. Collaborate with the Development/Business teams to understand the requirements. Analyzing Business Requirements/cases and developing/managing the positive/negative test scenarios cases....

  • Senior Engineer

    7 days ago

    india NTT DATA Full time

    Job Description Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive. Your day at...

  • Junior Security Engineer

    4 days ago

    india NTT DATA Full time

    Job Description Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive. Your day at...

  • Tableau Developer

    1 week ago

    India The Data Family Full time

    Company DescriptionThe Data Family offers analytics and data visualization services designed to transform data into actionable insights for our clients. Through our service, customers can access expert analytics, robust data engineering, and captivating visualizations to drive better decision-making and uncover opportunities for growth. Our primary customers...

  • Tableau Developer

    1 week ago

    India The Data Family Full time

    Company Description The Data Family offers analytics and data visualization services designed to transform data into actionable insights for our clients. Through our service, customers can access expert analytics, robust data engineering, and captivating visualizations to drive better decision-making and uncover opportunities for growth. Our primary...

  • Specialist - Data center Operations

    2 weeks ago

    India NTT DATA Full time

    Job Description NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future. In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our...

  • Data Engineer

    6 days ago

    india krtrimaIQ Cognitive Solutions Full time

    Company Description krtrimaIQ Cognitive Solutions is a technology company specializing in Cognitive Process Automation, Data Engineering, Statistical Analysis and Optimization, Data Science, Natural Language Processing, and Computer Vision. Our expertise allows us to transform organizations into intelligent enterprises. Role Description This is a full-time...

  • Data Engineer

    7 days ago

    India krtrimaIQ Cognitive Solutions Full time

    Company DescriptionkrtrimaIQ Cognitive Solutions is a technology company specializing in Cognitive Process Automation, Data Engineering, Statistical Analysis and Optimization, Data Science, Natural Language Processing, and Computer Vision. Our expertise allows us to transform organizations into intelligent enterprises.Role DescriptionThis is a full-time...

  • Data Engineer

    2 weeks ago

    India Paralleldots Full time

    Job Description : As a Senior Data Engineer at ParallelDots, you will be responsible for overseeing the design and development of advanced data solutions.You will work closely with data scientists, analysts, and other stakeholders to ensure that our data infrastructure meets the highest standards of scalability, reliability, and : Lead the design,...

  • Data Engineer

    2 weeks ago

    India Perfecthire Full time

    Job Description : We are seeking a talented and experienced Data Engineer to join our team and play a key role in the development and implementation of our end-to-end 834 EDI solution. A robust level of experience or familiarity with 834 EDI (Electronic Data Interchange) transactions is critical for this role. As a Data Engineer, you will be responsible...

  • Senior Engineer

    4 days ago

    india NTT DATA Full time

    Job Description Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive. Your day at...

  • Sr Microsoft Engineer

    4 days ago

    india NTT DATA Full time

    Job Description Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive. Your day at...

  • Data Engineer

    2 weeks ago

    India Lingaro Full time

    Role: Data Engineer – Lead ConsultantLocation: IndiaAbout Lingaro:Lingaro Group is the end-to-end data services partner to global brands and enterprises. We lead our clients through their data journey, from strategy through development to operations and adoption, helping them to realize the full value of their data.Since 2008, Lingaro has been recognized...

  • Data Engineer

    2 weeks ago

    India Lingaro Full time

    Role: Data Engineer – Lead Consultant Location: India About Lingaro: Lingaro Group is the end-to-end data services partner to global brands and enterprises. We lead our clients through their data journey, from strategy through development to operations and adoption, helping them to realize the full value of their data. Since 2008, Lingaro has been...