Data Engineer

7 days ago

Noida, Uttar Pradesh, India Cloudkeeper Full time ₹ 15,00,000 - ₹ 25,00,000 per year

Position Overview:

We are looking for an experienced and driven Data Engineer to join our team. The ideal candidate will have a strong foundation in big data technologies, particularly Spark, and a basic understanding of Scala to design and implement efficient data pipelines. As a Data Engineer at CloudKeeper, you will be responsible for building and maintaining robust data infrastructure, integrating large datasets, and ensuring seamless data flow for analytical and operational purposes.

Key Responsibilities:

  • Design, develop, and maintain scalable data pipelines and ETL processes to collect, process, and store data from various sources.
  • Work with Apache Spark to process large datasets in a distributed environment, ensuring optimal performance and scalability.
  • Develop and optimize Spark jobs and data transformations using Scala for large-scale data processing.
  • Collaborate with data analysts and other stakeholders to ensure data pipelines meet business and technical requirements.
  • Integrate data from different sources (databases, APIs, cloud storage, etc.) into a unified data platform.
  • Ensure data quality, consistency, and accuracy by building robust data validation and cleansing mechanisms.
  • Use cloud platforms (AWS, Azure, or GCP) to deploy and manage data processing and storage solutions.
  • Automate data workflows and tasks using appropriate tools and frameworks.
  • Monitor and troubleshoot data pipeline performance, optimizing for efficiency and cost-effectiveness.
  • Implement data security best practices, ensuring data privacy and compliance with industry standards.
  • Stay updated with new data engineering tools and technologies to continuously improve the data infrastructure.

Required Qualifications:

  • 4- 6 years of experience required as a Data Engineer or an equivalent role
  • Strong experience working with Apache Spark with Scala for distributed data processing and big data handling.
  • Basic knowledge of Python and its application in Spark for writing efficient data transformations and processing jobs.
  • Proficiency in SQL for querying and manipulating large datasets.
  • Experience with cloud data platforms, preferably AWS (e.g., S3, EC2, EMR, Redshift) or other cloud-based solutions.
  • Strong knowledge of data modeling, ETL processes, and data pipeline orchestration.
  • Familiarity with containerization (Docker) and cloud-native tools for deploying data solutions.
  • Knowledge of data warehousing concepts and experience with tools like AWS Redshift, Google BigQuery, or Snowflake is a plus.
  • Experience with version control systems such as Git.
  • Strong problem-solving abilities and a proactive approach to resolving technical challenges.
  • Excellent communication skills and the ability to work collaboratively within cross-functional teams.

Preferred Qualifications:

  • Experience with additional programming languages like Python, Java, or Scala for data engineering tasks.
  • Familiarity with orchestration tools like Apache Airflow, Luigi, or similar frameworks.
  • Basic understanding of data governance, security practices, and compliance regulations.

  • Data Engineer

    3 weeks ago

    Noida, Uttar Pradesh, India NTT DATA Full time

    Req ID 303115NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Data Engineer to join our team in Noida Uttar Pradesh IN-UP India IN Job Duties As a data engineer you will have a...

  • Mid Data Science

    3 weeks ago

    Noida, Uttar Pradesh, India NTT DATA Full time

    Req ID 312499 NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Mid Data Science AIML GenAI Lead Engineer to join our team in Noida Uttar Pradesh IN-UP India IN Job...

  • Systems Engineering Specialist

    2 weeks ago

    Noida, Uttar Pradesh, India NTT DATA Full time

    NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Systems Engineering Specialist to join our team At NTT DATA we know that with the right people on board anything is possible The...

  • Digital Engineering Sr. Engineer

    3 weeks ago

    Noida, Uttar Pradesh, India NTT Data Full time

    Job DescriptionNTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Sr. Engineer to join our team in Noida, Uttar Pradesh (IN-UP), India (IN).Job DescriptionAt NTT...

  • Network Engineering Advisor

    3 weeks ago

    Noida, Uttar Pradesh, India NTT DATA Full time

    Req ID 295196NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Network Engineering Advisor to join our team in Noida Uttar Pradesh IN-UP India IN At NTT DATA we know that with...

  • Devops Engineer

    3 weeks ago

    Noida, Uttar Pradesh, India NTT DATA Full time

    Req ID 291380NTT DATA Services strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a DevOps Engineer to join our team in Noida Uttar Pradesh IN-UP India IN Mandatory Skills Strong experience...

  • SQL Engineer

    3 weeks ago

    Noida, Uttar Pradesh, India NTT Data Full time

    Job DescriptionNTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a SQL Engineer to join our team in Noida, Uttar Pradesh (IN-UP), India (IN).Grade 8 - SQLJob DescriptionAt NTT DATA, we...

  • Data Engineer

    2 weeks ago

    Noida, Uttar Pradesh, India Coforge Full time

    Data Engineer - Informatica.Coforge Ltd. is seeking a Senior Data Engineer with a passion for crafting robust data pipelines to steer business strategies.If you excel in tackling intricate data hurdles and enjoy guiding others, this opportunity in Greater Noida could be your next career move.Job Location: Greater NoidaExperience: 5 to 7 YearsSend your CV to:...

  • Systems Engineering Advisor

    3 days ago

    Noida, Uttar Pradesh, India NTT DATA Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Req ID: 339179NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Systems Engineering Advisor to join our team in Noida, Uttar Pradesh (IN-UP), India (IN). Systems Engineering Advisor...

  • Data Engineer

    2 weeks ago

    Noida, Uttar Pradesh, India Coforge Full time

    Data Engineer - Informatica. Coforge Ltd. is seeking a Senior Data Engineer with a passion for crafting robust data pipelines to steer business strategies. If you excel in tackling intricate data hurdles and enjoy guiding others, this opportunity in Greater Noida could be your next career move. Job Location: Greater Noida Experience: 5 to 7...