Innovatily - Data Engineer - ETL Pipeline

2 weeks ago

Bangalore, India Innovatily Pvt. Ltd Full time

About Us :

Innovatily is dedicated to transforming data into actionable insights. We leverage cutting-edge technologies like Apache Spark and AWS Glue to build robust, scalable ETL pipelines that support our diverse data processing needs. We are seeking talented ETL Developers to join our team and help us scale our data infrastructure.

Job Description :

As an ETL Developer, you will be responsible for designing, building, and maintaining ETL pipelines using Apache Spark and AWS Glue. You will work closely with data engineers, analysts, and other stakeholders to ensure data is accessible, reliable, and processed efficiently.

Key Responsibilities :

- Design, develop, and deploy ETL pipelines using Apache Spark and AWS Glue.

- Collaborate with data engineers and analysts to understand data requirements and translate them into efficient ETL processes.

- Optimize data pipelines for performance, scalability, and robustness.

- Integrate data from various sources, including S3, relational databases, and APIs.

- Implement data validation and error handling mechanisms to ensure data quality.

- Monitor and troubleshoot ETL jobs to ensure data accuracy and pipeline reliability.

- Maintain and update existing data pipelines as data sources and requirements evolve.

- Document ETL processes, data models, and pipeline :

- Experience: 3+ years of experience in ETL development, with a focus on Apache Spark and AWS Glue.

Technical Skills :

- Strong expertise in Apache Spark (PySpark, Spark SQL) and AWS Glue.

- Proficiency in programming languages such as Python or Scala.

- Hands-on experience with AWS services like S3, Lambda, and IAM.

- Solid understanding of data warehousing concepts, data modeling, and ETL best practices.

- Familiarity with orchestration tools like Apache Airflow is a plus.

- Data Integration: Experience with integrating data from diverse sources, including relational databases, APIs, and flat files.

- Problem-Solving: Strong analytical and problem-solving skills with the ability to troubleshoot complex ETL issues.

- Communication: Excellent communication skills, with the ability to work collaboratively with cross-functional teams.

- Education: Bachelor's degree in computer science, Engineering, or a related field, or equivalent work experience.

Nice to Have :

- Experience with data lakes and big data processing.

- Familiarity with Glue Data Catalog, Glue Studio, and Glue Workflows.

- Knowledge of data governance and security practices in a cloud environment.

(ref:hirist.tech)

  • Innovatily | Data Engineer | bangalore

    3 weeks ago

    bangalore, India Innovatily Full time

    About Us:Innovatily is dedicated to transforming data into actionable insights. We leverage cutting-edge technologies like Apache Spark and AWS Glue to build robust, scalable ETL pipelines that support our diverse data processing needs. We are seeking talented ETL Developers to join our team and help us scale our data infrastructure.Job Description:As an ETL...

  • Innovatily | Data Engineer | bangalore

    3 weeks ago

    bangalore, India Innovatily Full time

    About Us: Innovatily is dedicated to transforming data into actionable insights. We leverage cutting-edge technologies like Apache Spark and AWS Glue to build robust, scalable ETL pipelines that support our diverse data processing needs. We are seeking talented ETL Developers to join our team and help us scale our data infrastructure. Job Description: As...

  • Data Engineer

    3 weeks ago

    Bangalore, India Innovatily Full time

    About Us: Innovatily is dedicated to transforming data into actionable insights. We leverage cutting-edge technologies like Apache Spark and AWS Glue to build robust, scalable ETL pipelines that support our diverse data processing needs. We are seeking talented ETL Developers to join our team and help us scale our data infrastructure. Job Description:...

  • Data Engineer

    3 weeks ago

    Bangalore, India Innovatily Full time

    About Us: Innovatily is dedicated to transforming data into actionable insights. We leverage cutting-edge technologies like Apache Spark and AWS Glue to build robust, scalable ETL pipelines that support our diverse data processing needs. We are seeking talented ETL Developers to join our team and help us scale our data infrastructure. Job Description:...

  • Innovatily - Technical Lead - Python Development

    4 weeks ago

    Bangalore, India Innovatily Pvt. Ltd Full time

    Innovatily is looking to hire a Technical Lead specializing in Data Applications and Python technology. Your role will be pivotal in implementing robust, scalable, and efficient solutions for data-centric applications. Responsibilities : - Technology Expert who possess strong expertise in data-related technologies, including data modelling, data integration,...

  • Innovatily | Inside Sales Executive | bangalore

    1 month ago

    bangalore, India Innovatily Full time

    About Innovatily: Innovatily is dedicated to delivering innovative solutions that empower our clients to excel in their respective markets. We specialize in building scalable, efficient, and modern systems that meet the highest standards of quality and performance. Position Summary: We are seeking an experienced Inside Sales Executives to join our dynamic...

  • Innovatily | Inside Sales Executive | bangalore

    1 month ago

    bangalore, India Innovatily Full time

    About Innovatily: Innovatily is dedicated to delivering innovative solutions that empower our clients to excel in their respective markets. We specialize in building scalable, efficient, and modern systems that meet the highest standards of quality and performance. Position Summary: We are seeking an experienced Inside Sales Executives to join our dynamic...

  • Innovatily | Marketing Manager | bangalore

    1 month ago

    bangalore, India Innovatily Full time

    Innovatily is seeking a dynamic and experienced Marketing Manager with a proven track record of success in global markets. The ideal candidate will be responsible for developing and implementing strategic marketing plans to expand our brand presence and drive revenue growth on a global scale. This role requires a deep understanding of international markets,...

  • Innovatily | Marketing Manager | bangalore

    1 month ago

    bangalore, India Innovatily Full time

    Innovatily is seeking a dynamic and experienced Marketing Manager with a proven track record of success in global markets. The ideal candidate will be responsible for developing and implementing strategic marketing plans to expand our brand presence and drive revenue growth on a global scale. This role requires a deep understanding of international markets,...

  • Innovatily - Python Developer - Artificial Intelligence/Machine Learning

    2 weeks ago

    Bangalore, India Innovatily Pvt. Ltd Full time

    About us : Innovatily is seeking a highly skilled and experienced Python Developer with a strong hands-on experience in Machine Learning/AI. As part of the team, you will play a crucial role in driving data-driven decision-making and developing cutting-edge artificial intelligence solutions. Your expertise in Python will contribute to the development of...

  • ETL Developer

    2 months ago

    Bangalore, India ProPMO Services Private Limited Full time

    Position : ETL Developer Location : Bangalore Experience Required : 5+ years Must-Have Skills : AWS Glue, Athena, S3, Redshift, Data Pipeline, Lambda, MySQL, SQL Server Position Overview : We are seeking an experienced ETL Developer with over 5 years of expertise to join our team in Bangalore . The ideal candidate will have a strong background in AWS...

  • Inside Sales Executive

    1 month ago

    Bangalore, India Innovatily Full time

    About Innovatily: Innovatily is dedicated to delivering innovative solutions that empower our clients to excel in their respective markets. We specialize in building scalable, efficient, and modern systems that meet the highest standards of quality and performance. Position Summary: We are seeking an experienced Inside Sales Executives to join...

  • Junior Data Engineer

    4 weeks ago

    Chennai/Bangalore, India Selsoft Full time

    Job role : Junior data Engineer Location : Chennai / Bangalore Experience : 1-2 years Job Description : - Building and maintaining data pipelines and ETL processes - Managing data storage and ensuring data quality - Developing and maintaining databases and data warehouses - Working with data scientists and analysts to understand business...

  • Data Engineer

    1 day ago

    Bangalore/Pune, India Talentpro Edge Full time

    About the RoleWe are seeking a skilled Data Engineer to join our team at Talentpro Edge as a Pipeline Developer. As a seasoned professional, you will play a pivotal part in designing and implementing large-scale data pipelines using Google Cloud Platform (GCP) services. Your expertise in ETL logic, data structures, and scripting languages such as Python will...

  • Data Engineer

    1 day ago

    Bangalore, India Yellow.ai Full time

    Responsibilities : Data Pipeline Development : - Assist in designing and building scalable ETL/ELT pipelines using tools. - Participate in the development and maintenance of data processing workflows, ensuring data accuracy and reliability. - Collaborate with the team to implement real-time and batch data solutions that meet business requirements. ...

  • Senior Data Engineer

    4 weeks ago

    Bangalore/Karnataka, India Kloud9 Full time

    Job Description : As a Senior Data Engineer, you will partner with Software Engineers, Data Scientists and Business Intelligence Engineers. You will gain a deep understanding of our services and the data they produce and become our resident expert in how to transform that data into a format that is useful for analytics and business intelligence. You will...

  • Senior Data Engineer

    4 weeks ago

    Bangalore, India ScaleOrange Technologies Pvt Ltd Full time

    Company Description : ScaleOrange Technologies Pvt Ltd. Role Description : This is a full-time on-site role for a Senior Data Engineer with 5+ years of experience at ScaleOrange Technologies Pvt Ltd in Hyderabad. The Senior Data Engineer will be responsible for data engineering, data modeling, ETL processes, data warehousing, and data analytics on a...

  • Senior Data Engineer

    1 day ago

    Bangalore, India Amicon Hub Services Full time

    At QSpark Group Inc, we rely on insightful data to power our systems and solutions. As a Data Engineer you will be working with the core team to build data pipelines, data analytics and helping our clients derive value from their data.We all know that data is hot and that companies need to use the data to come to insights. But do they know who is using their...

  • Data Engineer

    3 weeks ago

    Bangalore, India talentxo Full time

    Role & Responsibilities : - Design, develop, and maintain scalable data pipelines to extract, transform, and load (ETL) data from various sources into our data warehouse- Optimize and tune SQL queries for improved performance and efficiency- Collaborate with data scientists, analysts, and product managers to understand data requirements and deliver...

  • Data Engineer

    2 weeks ago

    Bangalore/Gurgaon/Gurugram, India InstaTalent Recruit LLP Full time

    Data Engineer (Snowflake). Required : Insta Talent Reruit LLP, a leading company in the None industry, is seeking a skilled Data Engineer to join our dynamic team. In this role, you will be responsible for building and running the data pipelines and services that are required to support business functions, reports, and dashboards. We heavily rely on...