Data Engineer

4 weeks ago

Bengaluru, India Innovatily Full time

About Us:


Innovatily is dedicated to transforming data into actionable insights. We leverage cutting-edge technologies like Apache Spark and AWS Glue to build robust, scalable ETL pipelines that support our diverse data processing needs. We are seeking talented ETL Developers to join our team and help us scale our data infrastructure.


Job Description:

As an ETL Developer, you will be responsible for designing, building, and maintaining ETL pipelines using Apache Spark and AWS Glue. You will work closely with data engineers, analysts, and other stakeholders to ensure data is accessible, reliable, and processed efficiently.


Key Responsibilities:

  • Design, develop, and deploy ETL pipelines using Apache Spark and AWS Glue.
  • Have strong python programing hands on knowledge.
  • Collaborate with data engineers and analysts to understand data requirements and translate them into efficient ETL processes.
  • Optimize data pipelines for performance, scalability, and robustness.
  • Integrate data from various sources, including S3, relational databases, and APIs.
  • Implement data validation and error handling mechanisms to ensure data quality.
  • Monitor and troubleshoot ETL jobs to ensure data accuracy and pipeline reliability.
  • Maintain and update existing data pipelines as data sources and requirements evolve.
  • Document ETL processes, data models, and pipeline configurations.


Qualifications:

  • Experience: 3+ years of experience in ETL development, with a focus on Apache Spark and AWS Glue.
  • Technical Skills:
  • Strong expertise in Apache Spark (PySpark, Spark SQL) and AWS Glue.
  • Proficiency in programming languages such as Python or Scala.
  • Hands-on experience with AWS services like S3, Lambda, and IAM.
  • Solid understanding of data warehousing concepts, data modeling, and ETL best practices.
  • Familiarity with orchestration tools like Apache Airflow is a plus.
  • Data Integration: Experience with integrating data from diverse sources, including relational databases, APIs, and flat files.
  • Problem-Solving: Strong analytical and problem-solving skills with the ability to troubleshoot complex ETL issues.
  • Communication: Excellent communication skills, with the ability to work collaboratively with cross-functional teams.
  • Education: Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent work experience.


Nice to Have:

  • Experience with data lakes and big data processing.
  • Familiarity with Glue Data Catalog, Glue Studio, and Glue Workflows.
  • Knowledge of data governance and security practices in a cloud environment.


Why Join Us?

  • Opportunity to work on challenging data projects with a talented team.
  • Competitive salary and benefits.
  • A culture of continuous learning and professional development.
  • Flexible working arrangements, including remote work options.


How to Apply:

If you are passionate about building robust ETL pipelines and want to make a significant impact, we would love to hear from you. Please send your resume and a brief cover letter outlining your relevant experience to Talent@innovatily.com.

  • Data Engineering Director

    2 weeks ago

    Bengaluru, Karnataka, India HEVO DATA Full time

    Engineering Leadership OpportunityThis is a unique chance to own and build scalable platform services that support thousands of customers' data pipelines, ensuring load data reliability to cloud warehouses.The Hevo platform core services will provide no-code-low pipeline building capabilities, making a significant impact on the company's business.About the...

  • Data Engineering Manager Lead

    2 weeks ago

    Bengaluru, Karnataka, India HEVO DATA Full time

    Scale Data Pipeline Services with HevoWe are seeking an experienced Data Engineering Manager to lead the development of scalable platform services supporting thousands of customers' data pipelines.This role offers a unique opportunity to build and own critical components of our cloud warehouse solution, ensuring reliable data loading and streamlining...

  • Data Engineering Lead with Scalable Platforms

    2 weeks ago

    Bengaluru, Karnataka, India HEVO DATA Full time

    Unlock Career Growth as a Data Engineering LeadWe're seeking an exceptional Data Engineering Lead to join our team at Hevo Data. In this role, you'll have the opportunity to own and build scalable platform services that support thousands of customers' data pipelines and ensure load data reliability to cloud warehouses.The Hevo platform core services will...

  • Chief Data Engineering Lead

    2 weeks ago

    Bengaluru, Karnataka, India Data Inc Full time

    About the RoleWe are seeking a highly skilled Chief Data Engineering Lead to join our team at Data Inc.This is an exciting opportunity to lead the development and implementation of architecture strategy and standards pertinent to the domain, and establish Standards and Best practices.

  • Hevo Data

    2 months ago

    Bengaluru, India HEVO DATA Full time

    This role provides a great opportunity to own and build highly scalable and fault-tolerant platform services that will help support thousands of customers' data pipelines and load data reliability to the different cloud warehouses. These services are the core of the Hevo platform that helps provide building no code-low pipeline. Also, makes a huge...

  • Data Engineer for Scalable Enterprise Systems

    4 days ago

    Bengaluru, Karnataka, India HEVO DATA Full time

    About Hevo DataWe empower businesses to drive growth by consolidating data from multiple sources, enabling faster analytics and informed decision making.Mission and VisionOur mission is to build technology that's simple to adopt and easy to access. We aim to unlock the potential of data for everyone.A Day in the Life of a Data Engineer at HevoYou'll work on...

  • Data engineer

    2 months ago

    Bengaluru, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data EngineerLocation: Yemalur, BangaloreYour Key Responsibilities:Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.Lead the architecture, design, and delivery of enterprise grade microservices.Deliver assigned projects within budgeted time, cost and agreed...

  • Data Engineer

    2 months ago

    Bengaluru, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data EngineerLocation: Yemalur, BangaloreYour Key Responsibilities:Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.Lead the architecture, design, and delivery of enterprise grade microservices.Deliver assigned projects within budgeted time, cost and agreed...

  • Data Engineer

    3 months ago

    Bengaluru, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost and agreed...

  • Data Engineer

    3 months ago

    Bengaluru, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost and agreed...

  • Lead Data Engineering Manager

    4 weeks ago

    Bengaluru, Karnataka, India Data Labs - LANDMARK GROUP Full time

    About UsData Labs - LANDMARK GROUP is a leading data engineering firm committed to delivering cutting-edge analytics solutions.Job DescriptionWe are seeking an experienced Lead Data Engineering Manager to oversee the development and deployment of our analytics products. This role involves leading a team of engineers in designing, building, and deploying...

  • Data Insights Specialist

    3 weeks ago

    Bengaluru, Karnataka, India Tatras Data Full time

    Become a key player in Tatras Data's journey to revolutionize Data Science and AI. As a Data Analyst, you will contribute to developing innovative solutions that drive business outcomes.Our team is located in Bangalore/Hyderabad, with a hybrid work model allowing for flexibility. We welcome professionals with 3+ years of experience and a strong educational...

  • Senior Data Engineering Manager

    4 weeks ago

    Bengaluru, Karnataka, India Data Labs - LANDMARK GROUP Full time

    About the RoleWe are seeking a Senior Data Engineering Manager to lead our engineering team in successfully building and deploying cutting-edge analytics products for consumption by various stakeholders.As a key member of our team, you will be responsible for leading the architecture, design, and delivery of enterprise-grade microservices. You will also...

  • Data Engineering Leader

    1 day ago

    Bengaluru, Karnataka, India Data Inc Full time

    Job SummaryWe are seeking a highly skilled and experienced Data Architect to lead our team at Data Inc. This is a key role in shaping the future of our data architecture and driving business growth.About the RoleThe successful candidate will be responsible for overseeing the development and implementation of our data architecture strategy, ensuring alignment...

  • Tatras Data | Data Analyst

    4 weeks ago

    Bengaluru, India Tatras Data Full time

    Tatras Data, an emerging player in Data Science and Artificial Intelligence is looking to hire multiple dataanalysts.Location: Bangalore/Hyderabad (Hybrid – 3 days in office)Experience Required: 3+ relevant years of experienceEducational Qualification:B.Tech, B.Sc., or Software EngineeringKey Skills and Requirements:● Programming Expertise: Strong...

  • Data Engineering Team Lead

    3 days ago

    Bengaluru, Karnataka, India Data Labs - LANDMARK GROUP Full time

    Data Labs - LANDMARK GROUP is seeking an experienced Data Engineer to lead our team in building and deploying cutting-edge analytics products.About the RoleWe are looking for a highly skilled Data Engineer with 8+ years of experience, including 5+ years in architecting and/or developing end-to-end products/solutions in the Analytics domain. The ideal...

  • Data analyst

    4 weeks ago

    Bengaluru, India Tatras Data Full time

    Tatras Data, an emerging player in Data Science and Artificial Intelligence is looking to hire multiple dataanalysts.Location: Bangalore/Hyderabad (Hybrid – 3 days in office)Experience Required: 3+ relevant years of experienceEducational Qualification: B. Tech, B. Sc., or Software EngineeringKey Skills and Requirements:● Programming Expertise: Strong...

  • Data Insights Professional

    4 weeks ago

    Bengaluru, Karnataka, India Tatras Data Full time

    Tatras Data, a pioneering player in the realm of Data Science and Artificial Intelligence, is seeking to recruit multiple professionals with expertise in data analysis.Job Type: Full-timeLocation: Bangalore/Hyderabad (Hybrid – 3 days in office)Estimated Salary Range: ₹18,00,000 - ₹25,00,000 per annumThe ideal candidate should possess at least 3+ years...

  • Data Insights Specialist

    3 weeks ago

    Bengaluru, Karnataka, India Tatras Data Full time

    Tatras Data is an emerging leader in Data Science and Artificial Intelligence, offering exciting opportunities for data enthusiasts.Estimated salary: ₹1,300,000 - ₹2,000,000 per annum (depending on experience).Job OverviewThis role involves analyzing complex data, identifying patterns, and providing actionable insights to drive business growth. If you...

  • (15h Left) Data Engineer

    2 months ago

    Bengaluru, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data EngineerLocation: Yemalur, BangaloreYour Key Responsibilities:Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.- Lead the architecture, design, and delivery of enterprise grade microservices.- Deliver assigned projects within budgeted time, cost and agreed...