Innovatily - Data Engineer - ETL Pipeline

1 month ago

Bengaluru, India Innovatily Pvt. Ltd Full time

About Us :

Innovatily is dedicated to transforming data into actionable insights. We leverage cutting-edge technologies like Apache Spark and AWS Glue to build robust, scalable ETL pipelines that support our diverse data processing needs. We are seeking talented ETL Developers to join our team and help us scale our data infrastructure.

Job Description :

As an ETL Developer, you will be responsible for designing, building, and maintaining ETL pipelines using Apache Spark and AWS Glue. You will work closely with data engineers, analysts, and other stakeholders to ensure data is accessible, reliable, and processed efficiently.

Key Responsibilities :

- Design, develop, and deploy ETL pipelines using Apache Spark and AWS Glue.

- Collaborate with data engineers and analysts to understand data requirements and translate them into efficient ETL processes.

- Optimize data pipelines for performance, scalability, and robustness.

- Integrate data from various sources, including S3, relational databases, and APIs.

- Implement data validation and error handling mechanisms to ensure data quality.

- Monitor and troubleshoot ETL jobs to ensure data accuracy and pipeline reliability.

- Maintain and update existing data pipelines as data sources and requirements evolve.

- Document ETL processes, data models, and pipeline :

- Experience: 3+ years of experience in ETL development, with a focus on Apache Spark and AWS Glue.

Technical Skills :

- Strong expertise in Apache Spark (PySpark, Spark SQL) and AWS Glue.

- Proficiency in programming languages such as Python or Scala.

- Hands-on experience with AWS services like S3, Lambda, and IAM.

- Solid understanding of data warehousing concepts, data modeling, and ETL best practices.

- Familiarity with orchestration tools like Apache Airflow is a plus.

- Data Integration: Experience with integrating data from diverse sources, including relational databases, APIs, and flat files.

- Problem-Solving: Strong analytical and problem-solving skills with the ability to troubleshoot complex ETL issues.

- Communication: Excellent communication skills, with the ability to work collaboratively with cross-functional teams.

- Education: Bachelor's degree in computer science, Engineering, or a related field, or equivalent work experience.

Nice to Have :

- Experience with data lakes and big data processing.

- Familiarity with Glue Data Catalog, Glue Studio, and Glue Workflows.

- Knowledge of data governance and security practices in a cloud environment.

(ref:hirist.tech)

  • Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Innovatily Pvt. Ltd Full time

    About Us :Innovatily is a data-driven company that leverages cutting-edge technologies to build robust, scalable ETL pipelines. We are seeking a talented ETL Developer to join our team and help us scale our data infrastructure.Job Description :As an ETL Developer, you will be responsible for designing, building, and maintaining ETL pipelines using Apache...

  • Senior Data Engineer Specialist

    5 days ago

    Bengaluru, Karnataka, India Innovatily Full time

    Transforming data into actionable insights is at the heart of Innovatily's mission. We harness cutting-edge technologies like Apache Spark and AWS Glue to craft robust, scalable ETL pipelines that meet our diverse data processing needs.We are seeking a talented Senior ETL Developer to join our team and help us scale our data infrastructure.About...

  • Data engineer

    5 days ago

    Bengaluru, India Innovatily Full time

    About Us:Innovatily is dedicated to transforming data into actionable insights. We leverage cutting-edge technologies like Apache Spark and AWS Glue to build robust, scalable ETL pipelines that support our diverse data processing needs. We are seeking talented ETL Developers to join our team and help us scale our data infrastructure.Job Description:As an ETL...

  • Data Engineer Specialist

    3 days ago

    Bengaluru, Karnataka, India Innovatily Full time

    Job DescriptionAbout Us:Innovatily is a leading data-driven company dedicated to transforming raw data into actionable insights. Our team leverages cutting-edge technologies like Apache Spark and AWS Glue to build robust, scalable data pipelines that meet our diverse data processing needs.We are seeking highly skilled ETL Developers to join our team and...

  • Data Engineer

    6 days ago

    Bengaluru, India Innovatily Full time

    About Us:Innovatily is dedicated to transforming data into actionable insights. We leverage cutting-edge technologies like Apache Spark and AWS Glue to build robust, scalable ETL pipelines that support our diverse data processing needs. We are seeking talented ETL Developers to join our team and help us scale our data infrastructure.Job Description:As an ETL...

  • Data Engineer

    5 days ago

    Bengaluru, India Innovatily Full time

    About Us:Innovatily is dedicated to transforming data into actionable insights. We leverage cutting-edge technologies like Apache Spark and AWS Glue to build robust, scalable ETL pipelines that support our diverse data processing needs. We are seeking talented ETL Developers to join our team and help us scale our data infrastructure.Job Description:As an ETL...

  • Data Engineer

    6 days ago

    Bengaluru, India Innovatily Full time

    About Us: Innovatily is dedicated to transforming data into actionable insights. We leverage cutting-edge technologies like Apache Spark and AWS Glue to build robust, scalable ETL pipelines that support our diverse data processing needs. We are seeking talented ETL Developers to join our team and help us scale our data infrastructure. Job Description: As...

  • Innovatily - Technical Lead - Python Development

    3 months ago

    Bengaluru, India Innovatily Pvt. Ltd Full time

    Innovatily is looking to hire a Technical Lead specializing in Data Applications and Python technology. Your role will be pivotal in implementing robust, scalable, and efficient solutions for data-centric applications. Responsibilities : - Technology Expert who possess strong expertise in data-related technologies, including data modelling, data integration,...

  • Data Applications Technical Lead

    5 days ago

    Bengaluru, Karnataka, India Innovatily Pvt. Ltd Full time

    About the Job Innovatily Pvt. Ltd is seeking a highly skilled Technical Lead to oversee the development of data-centric applications using Python technology. Responsibilities Provide technical leadership and mentorship to the development teamCollaborate with cross-functional teams to ensure seamless integration of components and technologiesLead in the...

  • Innovatily - Technical Content Writer - SEO Domain

    3 months ago

    Bengaluru, India Innovatily Private Limited Full time

    Company - InnovatilyJob Opening - TECHNICAL CONTENT WRITER Immediate Joining Work Mode - WFO- Innovatily is looking out for a talented and detail-oriented Technical Content Writer to join our team.- The ideal candidate Innovatily is looking out should possess a strong technical background, exceptional writing skills, and the ability to translate complex...

  • Innovatily - Marketing Manager

    2 weeks ago

    Bengaluru, India Innovatily Private Limited Full time

    Innovatily is seeking a dynamic and experienced Marketing Manager with a proven track record of success in global markets. The ideal candidate will be responsible for developing and implementing strategic marketing plans to expand our brand presence and drive revenue growth on a global scale. This role requires a deep understanding of international markets,...

  • Machine Learning Engineer

    4 days ago

    Bengaluru, Karnataka, India Innovatily Pvt. Ltd Full time

    About Innovatily Pvt. Ltd.Innovatily Pvt. Ltd is a company seeking an experienced Machine Learning Engineer to join its team. The successful candidate will have a strong background in Python development and artificial intelligence/machine learning.Company Overview:Innovatily Pvt. Ltd is a leading provider of innovative solutions, leveraging the power of data...

  • Data Engineer for Cloud Migration and Data Pipelining Expert

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team in Bangalore, India. This role involves designing, building, and maintaining scalable data pipelines using Databricks on AWS.The ideal candidate will have experience with Databricks, including Apache Spark development, and proficiency in Python or Scala for Spark job development....

  • Data Pipeline Engineer

    1 week ago

    Bengaluru, Karnataka, India PRI Global Full time

    About PRI GlobalPRI Global is a leading company in the field of data warehousing and ETL development.Job SummaryWe are seeking an experienced Data Pipeline Engineer to join our team. As a key member of our engineering department, you will be responsible for designing, developing, and maintaining SSIS data pipelines.Key ResponsibilitiesDesign and develop...

  • Innovatily - Python Developer - Artificial Intelligence/Machine Learning

    1 month ago

    Bengaluru, India Innovatily Pvt. Ltd Full time

    About us :Innovatily is seeking a highly skilled and experienced Python Developer with a strong hands-on experience in Machine Learning/AI. As part of the team, you will play a crucial role in driving data-driven decision-making and developing cutting-edge artificial intelligence solutions. Your expertise in Python will contribute to the development of...

  • Inside Sales Executive

    3 weeks ago

    Bengaluru, India Innovatily Full time

    About Innovatily:Innovatily is dedicated to delivering innovative solutions that empower our clients to excel in their respective markets. We specialize in building scalable, efficient, and modern systems that meet the highest standards of quality and performance.Position Summary:We are seeking an experienced Inside Sales Executives to join our dynamic sales...

  • Senior Software Engineer

    4 weeks ago

    Bengaluru, Karnataka, India Innovatily Full time

    Company Overview:Innovatily is a technology-driven organization that strives for excellence in software development.About the Role:We are seeking a highly skilled C++ Developer to join our team. The ideal candidate will be responsible for developing high-quality software applications and designing highly scalable, testable code.Key Responsibilities:Develop...

  • Sales Growth Specialist

    4 days ago

    Bengaluru, Karnataka, India Innovatily Full time

    About InnovatilyInnovatily is dedicated to delivering cutting-edge solutions that empower our clients to excel in their respective markets. We specialize in building scalable, efficient, and modern systems that meet the highest standards of quality and performance.Job DescriptionWe are seeking an experienced Sales Growth Specialist to join our dynamic sales...

  • Inside sales executive

    2 weeks ago

    Bengaluru, India Innovatily Full time

    About Innovatily:Innovatily is dedicated to delivering innovative solutions that empower our clients to excel in their respective markets. We specialize in building scalable, efficient, and modern systems that meet the highest standards of quality and performance.Position Summary:We are seeking an experienced Inside Sales Executives to join our dynamic sales...

  • Junior Data Engineer

    3 months ago

    Bengaluru, India Selsoft Full time

    Job role : Junior data EngineerLocation : Chennai / BangaloreExperience : 1-2 yearsJob Description : - Building and maintaining data pipelines and ETL processes - Managing data storage and ensuring data quality - Developing and maintaining databases and data warehouses - Working with data scientists and analysts to understand business requirements and...