Data Engineer

4 weeks ago

Bengaluru, Karnataka, India Rapsys Technologies Full time

Responsibilities :

- Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration.

- Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.

- Implement data integration and transformation processes to ensure optimal performance and reliability of data pipelines.

- Optimize and fine-tune existing data pipelines / Airflow to improve efficiency, scalability, and Troubleshoot and resolve issues related to data pipelines, ensuring smooth operation and minimal downtime.

- Work closely with AWS services like S3, Glue, EMR, Redshift, and other related technologies to design and optimize data infrastructure.

- Develop and maintain documentation for data pipelines, processes, and system architecture.

- Stay updated with the latest industry trends and best practices related to data engineering and AWS services.

Requirements :

- Bachelors degree in Computer Science, Engineering, or a related field.

- Proficiency in Python, PySpark and SQL for data processing and manipulation.

- Min 5 years of experience in data engineering, specifically working with Apache Airflow and AWS technologies.

- Strong knowledge of AWS services, particularly S3, Glue, EMR, Redshift, and AWS Lambda.

- Understanding of Snowflake Data Lake is preferred.

- Experience with optimizing and scaling data pipelines for performance and efficiency.

- Good understanding of data modeling, ETL processes, and data warehousing concepts.

- Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment.

- Effective communication skills and the ability to articulate technical concepts to non-technical stakeholders.

Preferred Qualifications :

- AWS certification(s) related to data engineering or big data.

- Experience working with big data technologies like Snowflake, Spark, Hadoop, or related frameworks.

- Familiarity with other data orchestration tools in addition to Apache Airflow.

- Knowledge of version control systems like Bitbucket, Git.

(ref:hirist.tech)

  • Data Engineer

    6 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack. Provide thought leadership by recommending the most appropriate technologies and solutions for a given use case, covering the entire spectrum from the application layer...

  • Data Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Migrate ETL workflows from SAP BODS to AWS Glue/dbt/Talend. Develop and maintain scalable ETL pipelines in AWS. Write PySpark scripts for large-scale data processing. Optimize SQL queries and transformations for AWS PostgreSQL. Work with Cloud Engineers to ensure smooth deployment and performance tuning. Integrate data pipelines with existing Unix systems....

  • Data Engineer

    6 days ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Req ID:321800We are currently seeking a Data Engineer (Talend &Pyspark) to join our team in Bangalore, Karntaka (IN-KA), India (IN)."Job Duties: Key Responsibilities: Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical...

  • Data Engineer

    6 days ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Key Responsibilities:Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack.Provide thought leadership by recommending the most appropriate technologies and solutions for a given use case, covering the entire spectrum from...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Req ID: 338632NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer (HRIS,Tableau and BI) to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Job Duties: Key...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time US$ 90,000 - US$ 1,20,000 per year

    Req ID: 338630NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer (HRIS) to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Job Duties: Key...

  • Data Engineer

    6 days ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    We are currently seeking a Data Engineer (HRIS) to join our team in Bangalore, Karntaka (IN-KA), India (IN).Job Duties: Key Responsibilities -Data Management - Develop data mapping specifications and transformation rules Perform data cleansing, validation, and reconciliation activities Create and execute data conversion scripts and processes Document data...

  • Sr Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Req ID: 340286NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Sr Data Engineer (Life and Annuity) to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Job Duties: At...

  • Senior Data Engineer cum Data Scientist

    6 days ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About the RoleAs a Senior Data Engineer cum Data Scientist, you will be a key contributor to designing, developing, and operating advanced data architectures, pipelines, and analytics solutions within the Palantir ecosystemYou will leverage your deep expertise in Palantir Foundry and Gotham platforms, combined with strong AI capabilities, to solve complex...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Business Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Experience: 4 to 6 YearsLocation: Hybrid/Bangalore]Employment Type: Full-TimeRole & responsibilitiesMust-Have Skills:46 years of experience in data engineering roles.Proven experience in data migration and conversion projects (SAP or non-SAP).Strong knowledge of ETL concepts, tools, and frameworks.Proficiency in SQL, Python, or other data scripting...