Data Engineer with Apache Spark and AWS Experience

3 weeks ago


Chennai, Tamil Nadu, India Gainwell Technologies Full time

Job Summary


We are seeking a highly experienced data engineer with Apache Spark and AWS expertise to join our data analytics team at Gainwell Technologies. As a key member of our cross-functional team, you will have the opportunity to design, develop, and deploy large-scale data pipelines, leveraging your technical skills to drive business growth and success.


Your Role in Our Mission



  1. Design, develop, and deploy data pipelines, including ETL processes, using Apache Spark and AWS.

  2. Monitor, manage, validate, and test data extraction, movement, transformation, loading, normalization, cleansing, and updating processes.

  3. Collaborate with users to understand data needs and deliver data with a focus on data quality, reuse, consistency, security, and regulatory compliance.

  4. Collaborate with team members on data models and schemas in our data warehouse.

  5. Communicate effectively with various internal and external stakeholders.

Requirements




  • 15+ years of experience in data engineering or a related field.

  • Technical skills in SQL, Spark/Pyspark, Databricks, and Python.

  • Proven experience designing and building large-scale data pipelines.

  • In-depth knowledge of big data technologies, including Apache Spark, Kafka, and Hadoop.

  • Strong proficiency in SQL and scripting languages, such as Python and Bash.

  • Experience with cloud platforms, such as AWS, GCP, and Azure, is a plus.

What to Expect


A fast-paced, challenging, and rewarding work environment. A hybrid office environment. Late evening work may be required to overlap US work hours.


  • Data Engineer

    4 weeks ago


    Chennai, Tamil Nadu, India Gainwell Technologies Full time

    Data Engineer RoleWe are seeking a skilled Data Engineer to join our team at Gainwell Technologies. As a Data Engineer, you will be responsible for designing, developing, and deploying data pipelines using Apache Spark and AWS. Your expertise in big data technologies, such as Apache Spark, Kafka, and Hadoop, will be crucial in building large-scale data...

  • Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India Huntsmen and barons Full time

    Job SummaryKey ResponsibilitiesUtilize Apache Spark for developing data processing applications.Develop and maintain large-scale data processing pipelines.Collaborate with cross-functional teams to ensure data quality and integrity.Requirements4+ years of experience in data engineering and data processing.Strong proficiency in Apache Spark and Scala.In-depth...


  • Chennai, Tamil Nadu, India Frost and Sullivan (India) Private Limited Full time

    **Job Summary**Frost & Sullivan seeks a highly skilled Data Architect to join our Digital Products team. As a Data Architect at Frost & Sullivan, you will be responsible for designing, building, and maintaining our organization's data architecture.The ideal candidate will have experience in designing and implementing data models, data integration solutions,...

  • Lead Data Engineer

    1 month ago


    Chennai, Tamil Nadu, India 1HResource Solutions Full time

    Location : Chennai. Shift : 11AM 8 PM or 1PM 10PM. Skills : Python, Big-Data, spark/pyspark, databricks, ADF or ADL, team management experience.Role Description :- As a Technical Lead Big Data Platform, you will be responsible for designing and implementing big data platforms to support large-scale data integration projects.- You will work with data...

  • Senior Data Engineer

    4 weeks ago


    Chennai, Tamil Nadu, India Gainwell Technologies Full time

    Data Engineering RoleWe are seeking a seasoned Data Engineer to join our data analytics team at Gainwell Technologies. As a key member of our team, you will be responsible for designing, developing, and deploying data pipelines using Apache Spark and AWS. Your expertise in big data technologies, including Apache Spark, Kafka, and Hadoop, will be crucial in...

  • Senior Data Engineer

    2 months ago


    Chennai, Tamil Nadu, India Getinz Full time

    About the Role : We are looking for a highly skilled and motivated Senior Data Engineer to join our dynamic team. You will play a key role in designing, implementing, and optimizing data architectures and pipelines to support scalable data solutions for our business.Qualifications : - 5-8 years of experience in data engineering, with a focus on building and...

  • Senior Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India Gainwell Technologies Full time

    About this roleWe are seeking an experienced Data Engineer to join our data analytics team at Gainwell Technologies. As a key member of our team, you will design, develop, and deploy data pipelines using Apache Spark and AWS to support our analytic missions.Your responsibilitiesData pipeline design and developmentETL-processes for data extraction,...


  • Chennai, Tamil Nadu, India Gainwell Technologies Full time

    Data Engineering Role OverviewWe are seeking a highly skilled Data Engineer to join our team at Gainwell Technologies. As a Data Engineer, you will play a key role in designing, developing, and deploying data pipelines, including ETL processes, to extract, process, and deliver data using Apache Spark Framework.Key ResponsibilitiesDesign and Develop Data...


  • Chennai, Tamil Nadu, India WILLWARE TECHNOLOGIES Full time

    Job Title: AWS Cloud Data EngineerJob SummaryWe are seeking a highly skilled AWS Cloud Data Engineer to join our team at WILLWARE TECHNOLOGIES. The successful candidate will design, develop, and deploy complex data pipelines using Apache Airflow on AWS infrastructure. This is an excellent opportunity for a data engineer to work on challenging projects and...

  • AWS Data Engineer

    4 weeks ago


    Chennai, Tamil Nadu, India WILLWARE TECHNOLOGIES Full time

    Job Title: AWS Data Engineer - Pipeline SpecialistAbout the Role:As a key member of our data engineering team at WILLWARE TECHNOLOGIES, you will be responsible for designing, developing, and deploying complex data pipelines using Apache Airflow on AWS infrastructure. Your expertise in data engineering, combined with your passion for innovative solutions,...

  • Data Engineer

    1 month ago


    Chennai, Tamil Nadu, India Cloudsara Technologies Private Limited Full time

    Transforming Data Landscape at Cloudsara Technologies Private LimitedWe are seeking a talented Data Engineer to join our dynamic team in the Computer Software industry. This role is pivotal in transforming our data landscape, enabling our teams to leverage data for strategic insights. As a Data Engineer, you will work collaboratively with data scientists,...


  • Chennai, Tamil Nadu, India Fulcrum Digital Full time

    About the RoleFulcrum Digital is a cutting-edge digital transformation company providing innovative technology services to various industries. We are seeking a skilled Senior Data Engineer with expertise in AWS cloud technologies and data engineering.The ideal candidate will be proficient in a wide range of AWS services and tools, including Databricks, MWAA,...

  • Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India RiskInsight Consulting Pvt Ltd Full time

    Job Title: Data Engineer - Big Data ArchitectureJob Summary:We are seeking a highly skilled Data Engineer to design, develop, and deploy scalable Big Data applications using Apache Spark.The successful candidate will collaborate with data scientists and business analysts to understand requirements and translate them into technical solutions.The Data Engineer...


  • Chennai, Tamil Nadu, India Fulcrum Digital Full time

    We are seeking an experienced professional with a strong background in AWS cloud technologies and data engineering to join our team at Fulcrum Digital.The ideal candidate will be proficient in a wide range of AWS services and tools, including Databricks, MWAA, Lambda, and Snowflake, among others.This role involves developing, managing, and optimizing...


  • Chennai, Tamil Nadu, India Fulcrum Digital Full time

    About the RoleFulcrum Digital is a next-generation digital accelerating company providing digital transformation and technology services across various industries.We are seeking an experienced professional with a strong background in AWS cloud technologies and data engineering.The ideal candidate will be proficient in a wide range of AWS services and tools,...


  • Chennai, Tamil Nadu, India Getinz Techno Services Full time

    About the Role:We are seeking a highly skilled and motivated Senior Data Engineer to join our team at Getinz Techno Services. The ideal candidate will play a key role in designing, implementing, and optimizing data architectures and pipelines to support scalable data solutions for our business.Key Responsibilities:Design, build, and maintain efficient and...


  • Chennai, Tamil Nadu, India Fulcrum Digital Full time

    About the RoleFulcrum Digital is a leading digital transformation company that provides cutting-edge technology services. We are seeking an experienced professional with a strong background in AWS cloud technologies and data engineering.Key ResponsibilitiesDesign, develop, and maintain scalable data pipelines using AWS Databricks, AWS Lambda, AWS S3, Delta...

  • Data Engineer

    2 months ago


    chennai, Tamil Nadu, India MNR Solutions Full time

    Salary : 19-30lpaWe are seeking a skilled Data Engineer with 3 to 9 years of experience to join our team in Chennai. The ideal candidate will be responsible for designing, building, and maintaining data pipelines and data architecture to support data analytics and business intelligence initiatives. This role requires a strong understanding of data processing...


  • Chennai, Tamil Nadu, India Fulcrum Digital Full time

    At Fulcrum Digital, we are seeking a highly skilled professional with a strong background in AWS cloud technologies and data engineering to join our team.The ideal candidate will be proficient in a wide range of AWS services and tools, including Databricks, MWAA, Lambda, and Snowflake, among others.This role involves developing, managing, and optimizing...


  • Chennai, Tamil Nadu, India Fulcrum Digital Full time

    About Fulcrum DigitalWe are an agile and next-generation digital accelerating company providing digital transformation and technology services right from ideation to implementation. Our services have applicability across a variety of industries including banking & financial services, insurance, retail, higher education, food, healthcare, and...