Data Engineer

2 months ago

gurugram, India Material Full time

Role :- AWS Data Engineer


Location :- Gurgaon / Bangalore


Notice :- Immediate Joiners Only


Skills:- AWS , Spark , Python , Glue , S3 , Redshift etc..


Experience:- 4+ Years


Position Overview:

We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. The ideal candidate will have extensive experience with AWS Glue, Apache Airflow, Kafka, SQL, Python and DataOps tools and technologies. Knowledge of SAP HANA & Snowflake is a plus. This role is critical for designing, developing, and maintaining our client’s data pipeline architecture, ensuring the efficient and reliable flow of data across the organization.

Key Responsibilities:

  • Design, Develop, and Maintain Data Pipelines:
  • Develop robust and scalable data pipelines using AWS Glue, Apache Airflow, and other relevant technologies.
  • Integrate various data sources, including SAP HANA, Kafka, and SQL databases, to ensure seamless data flow and processing.
  • Optimize data pipelines for performance and reliability.
  • Data Management and Transformation:
  • Design and implement data transformation processes to clean, enrich, and structure data for analytical purposes.
  • Utilize SQL and Python for data extraction, transformation, and loading (ETL) tasks.
  • Ensure data quality and integrity through rigorous testing and validation processes.
  • Collaboration and Communication:
  • Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet their needs.
  • Collaborate with cross-functional teams to implement DataOps practices and improve data life cycle management.
  • Monitoring and Optimization:
  • Monitor data pipeline performance and implement improvements to enhance efficiency and reduce latency.
  • Troubleshoot and resolve data-related issues, ensuring minimal disruption to data workflows.
  • Implement and manage monitoring and alerting systems to proactively identify and address potential issues.
  • Documentation and Best Practices:
  • Maintain comprehensive documentation of data pipelines, transformations, and processes.
  • Adhere to best practices in data engineering, including code versioning, testing, and deployment procedures.
  • Stay up-to-date with the latest industry trends and technologies in data engineering and DataOps.

Required Skills and Qualifications:

  • Technical Expertise:
  • Extensive experience with AWS Glue for data integration and transformation.
  • Proficient in Apache Airflow for workflow orchestration.
  • Strong knowledge of Kafka for real-time data streaming and processing.
  • Advanced SQL skills for querying and managing relational databases.
  • Proficiency in Python for scripting and automation tasks.
  • Experience with SAP HANA for data storage and management.
  • Familiarity with Data Ops tools and methodologies for continuous integration and delivery in data engineering.
  • Preferred Skills:
  • Knowledge of Snowflake for cloud-based data warehousing solutions.
  • Experience with other AWS data services such as Redshift, S3, and Athena.
  • Familiarity with big data technologies such as Hadoop, Spark, and Hive.
  • Soft Skills:
  • Strong analytical and problem-solving skills.
  • Excellent communication and collaboration abilities.
  • Detail-oriented with a commitment to data quality and accuracy.
  • Ability to work independently and manage multiple projects simultaneously.

  • Digital Engineering Engineer

    3 weeks ago

    Gurugram, India NTT DATA Services Full time

    Req ID: 295081 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Engineer to join our team in Gurgaon, Haryāna (IN-HR), India (IN).Job Duties: Must have:...

  • Data Engineer

    1 month ago

    gurugram, India Turnberry Solutions Full time

    Experienced data engineer with hands on experience on GCP offerings Experienced in BigQuery/ BigTable/ PysparkWorked on prior data engineering projects leveraging GCP product offeringsStrong SQL backgroundPrior Amex Bigdata experience is big PlusData engineering pipeline build on big data on GCPSomeone who has worked with GCPStrong SQLStrong big...

  • Eucloid - Data Engineer - Azure Databricks

    6 months ago

    Gurgaon/Gurugram/Chennai, India Eucloid Data Solutions Pvt Ltd Full time

    Job Description - Data Engineer (Databricks)About Eucloid : Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for...

  • Senior Data Engineer

    4 weeks ago

    Chennai/Gurgaon/Gurugram/Delhi NCR/Hyderabad, India Eucloid Data Solutions Pvt Ltd Full time

    About Eucloid Data SolutionsEucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. We offer bespoke, full stack and data-driven approach to solve highly complex and large-scale business problems for some of the biggest Fortune 500...

  • Data Engineer

    6 days ago

    Gurugram, India StatusNeo Full time

    Job descriptionJob Title: Data EngineerCompany: StatusNeoAboutAt StatusNeo, we are on a mission to redefine the future of data-driven decision-making. We are looking for a passionate and talented Junior Data Engineer to join our dynamic team. If you have up to one year of experience or are fresh out of college with a strong academic background in data...

  • Data Engineer

    7 days ago

    Gurugram, India StatusNeo Full time

    Job descriptionJob Title: Data EngineerCompany: StatusNeoAboutAt StatusNeo, we are on a mission to redefine the future of data-driven decision-making. We are looking for a passionate and talented Junior Data Engineer to join our dynamic team. If you have up to one year of experience or are fresh out of college with a strong academic background in data...

  • Data Engineer

    7 days ago

    Gurugram, India StatusNeo Full time

    Job description Job Title: Data Engineer Company: StatusNeo About At StatusNeo, we are on a mission to redefine the future of data-driven decision-making. We are looking for a passionate and talented Junior Data Engineer to join our dynamic team. If you have up to one year of experience or are fresh out of college with a strong academic background in data...

  • Data Engineer

    1 month ago

    gurugram, India Nutrabay Full time

    We are seeking a skilled Data Engineer to join our team. The ideal candidate will be responsible for handling the movement of data from various sources to our data warehouses and data lakes. The candidate should have a strong understanding of data engineering concepts and experience working with databases such as Redshift, MySQL, and technologies like Athena...

  • Data Engineer/Senior Data Engineer

    6 days ago

    Gurugram, India Cognitio Analytics Inc Full time

    Experience: 1 to 8 years - DE/ Sr. DE/ AM /Manager Must Have: SQL Python/ Pyspark Azure Databricks, ADLS Gen 2 Blob Storage, Key Vaults, BitBucket, Version control, Azure DevOps ETL, Building Data Pipelines, Data Warehousing, Data Modelling and Governance, Data Security Agile Practices, SDLC Data Exploration, Data Quality metrics, Historical data...

  • Data Engineer/Senior Data Engineer

    7 days ago

    Gurugram, India Cognitio Analytics Inc Full time

    Experience:1 to 8 years - DE/ Sr. DE/ AM /ManagerMust Have:SQLPython/ Pyspark Azure Databricks, ADLS Gen 2 Blob Storage, Key Vaults, BitBucket, Version control, Azure DevOpsETL, Building Data Pipelines, Data Warehousing, Data Modelling and Governance, Data SecurityAgile Practices, SDLCData Exploration, Data Quality metrics, Historical data Management, Good...

  • Data Engineer

    1 month ago

    Gurugram, India Turnberry Solutions Full time

    Experienced data engineer with hands on experience on GCP offerings Experienced in BigQuery/ BigTable/ PysparkWorked on prior data engineering projects leveraging GCP product offeringsStrong SQL backgroundPrior Amex Bigdata experience is big PlusData engineering pipeline build on big data on GCPSomeone who has worked with GCPStrong SQLStrong big...

  • Data Engineer

    2 months ago

    gurugram, India Turnberry Solutions Full time

    Experienced data engineer with hands on experience on GCP offerings Experienced in BigQuery/ BigTable/ PysparkWorked on prior data engineering projects leveraging GCP product offeringsStrong SQL backgroundPrior Amex Bigdata experience is big PlusData engineering pipeline build on big data on GCPSomeone who has worked with GCPStrong SQLStrong big...

  • Data Engineer

    2 months ago

    Gurugram, India Turnberry Solutions Full time

    Experienced data engineer with hands on experience on GCP offerings Experienced in BigQuery/ BigTable/ PysparkWorked on prior data engineering projects leveraging GCP product offeringsStrong SQL backgroundPrior Amex Bigdata experience is big PlusData engineering pipeline build on big data on GCPSomeone who has worked with GCPStrong SQLStrong big...

  • Data Engineer

    2 months ago

    Gurugram, India Turnberry Solutions Full time

    Experienced data engineer with hands on experience on GCP offerings Experienced in BigQuery/ BigTable/ Pyspark Worked on prior data engineering projects leveraging GCP product offerings Strong SQL background Prior Amex Bigdata experience is big Plus Data engineering pipeline build on big data on GCP Someone who has worked with GCP Strong SQL Strong big query...

  • Data Engineer

    1 month ago

    gurugram, India Turnberry Solutions Full time

    Experienced data engineer with hands on experience on GCP offerings Experienced in BigQuery/ BigTable/ Pyspark Worked on prior data engineering projects leveraging GCP product offerings Strong SQL background Prior Amex Bigdata experience is big Plus Data engineering pipeline build on big data on GCP Someone who has worked with GCP Strong SQL Strong big query...

  • Data Engineer

    1 day ago

    Gurugram, India Aays Full time

    Position: Senior Data Engineer / Data Engineer Desired Experience: 3+ years You will act as a key member of the Data consulting team, working directly with the partners and senior stakeholders of the clients designing and implementing big data and analytics solutions. Communication and organisation skills are keys for this position, along with a...

  • Data Engineer

    12 hours ago

    Gurugram, India Aays Full time

    Position : Senior Data Engineer / Data Engineer Desired Experience : 3+ years You will act as a key member of the Data consulting team, working directly with the partners and senior stakeholders of the clients designing and implementing big data and analytics solutions. Communication and organisation skills are keys for this position, along with a...

  • Data Engineer

    2 days ago

    Gurugram, India Aays Full time

    Position: Senior Data Engineer / Data Engineer Desired Experience: 3+ years You will act as a key member of the Data consulting team, working directly with the partners and senior stakeholders of the clients designing and implementing big data and analytics solutions. Communication and organisation skills are keys for this position, along with a...

  • Data Engineer

    6 months ago

    Gurugram, India Nutrabay Full time

    We are seeking a skilled Data Engineer to join our team. The ideal candidate will be responsible for handling the movement of data from various sources to our data warehouses and data lakes. The candidate should have a strong understanding of data engineering concepts and experience working with databases such as Redshift, MySQL, and technologies like Athena...

  • Data Engineer

    3 weeks ago

    Gurugram, India Markovate Full time

    Job Summary We are seeking a skilled and motivated Data Engineer with a strong background in data integration, ETL processes, and data warehousing. The ideal candidate will have over 2 years of experience in data engineering, with a good understanding of data architecture, pipeline creation, and big data technologies. This role requires a proactive...