Data Engineer-aws, Python

2 days ago

Noida Uttar Pradesh, India 3Pillar Global Full time

Embark on an exciting journey into the realm of software development with 3Pillar We extend an invitation for you to join our team and gear up for a thrilling adventure At 3Pillar our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making As a Senior Data Engineer you will hold a crucial position within our dynamic team actively contributing to thrilling projects that reshape data analytics for our clients providing them with a competitive advantage in their respective industries Job Summary For this role we are seeking a Senior Data Engineer for our Client s ETL Support Team to act as a technical pillar during our Client s transition to an AWS-native data environment You will be a key escalation point responsible for handling the most complex data pipeline issues This role may move beyond simple support you might be requested to harden and improve our Client s systems by driving performance tuning cost optimization and automation initiatives Work Location Noida Sec 62 Working days per week 5 Rotational Shift Candidates should be flexible working on a Roster basis including Weekends but maximum 5 days a week Responsibilities Job Monitoring Monitor approximately 2 300 scheduled jobs daily weekly monthly to ensure timely and successful execution Execute on-demand jobs as required by the business Troubleshooting RCA Troubleshoot job failures perform detailed root cause analysis RCA and provide clear documentation for all findings Issue Resolution Address and resolve bugs and data-related issues reported by the business team Data Integrity Verify source file placement in designated directories to maintain data integrity Reload Change Data Capture CDC tables when structural changes occur in source systems Data Synchronization Help manage synchronization between external databases including Teradata write-backs and AWS Glue tables Development Assist in developing new solutions enhancements and bug fixes using AWS Glue Lambda and Step Functions Stakeholder Support Answer questions from the business and support User Acceptance Testing UAT inquiries Decision Making Make timely decisions to resolve issues execute tasks efficiently and escalate complex problems to senior or lead engineers as needed all while maintaining agreed-upon SLAs Documentation Participate in knowledge transfer to internal resources and assist in creating necessary documentation and runbooks Qualifications Experience 3-5 years of relevant experience Technical Skills In-depth knowledge of AWS Glue AWS Lambda and AWS Step Functions Core Concepts A deep understanding of ETL processes and data warehouse design Problem-Solving Proven ability to troubleshoot data pipelines and perform root cause analysis RCA Should have worked on Python scripting Experience Hands-on experience with Glue Lambda and Step Function development Availability Must be able to work a day shift that includes coverage for weekends and holidays on a rotational basis Preferred Skills Familiarity with Change Data Capture CDC processes Teradata and integration with SAS RMJ events

  • AWS Data Engineer

    2 weeks ago

    uttar pradesh, India Coforge Full time

    We are seeking a skilled Data Engineer (AWS- Scala Mandatory) to join our dynamic team. This role involves building and maintaining robust data pipelines, designing scalable databases, and collaborating with stakeholders to deliver high-quality data solutions.ResponsibilitiesBuild and maintain data pipelines using AWS Glue, Python, EMR/Spark, and...

  • Senior Data Engineer-aws, Python

    2 days ago

    Noida, Uttar Pradesh, India 3Pillar Global Full time

    Embark on an exciting journey into the realm of software development with 3Pillar We extend an invitation for you to join our team and gear up for a thrilling adventure At 3Pillar our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making As a Senior Data Engineer you will hold a crucial...

  • Data Engineer-AWS, Python

    3 days ago

    Noida, Uttar Pradesh, India 3Pillar Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Embark on an exciting journey into the realm of software development with 3Pillar We extend an invitation for you to join our team and gear up for a thrilling adventure. At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Senior Data Engineer, you will hold a crucial...

  • Lead Data Engineer

    2 weeks ago

    Noida, Uttar Pradesh, India Lumiq Full time

    Lead Data Engineer AWS Who we are LUMIQ is the leading Data and Analytics company in the Financial Services and Insurance FSI industry We are trusted by the world s largest FSIs including insurers banks AMCs and NBFCs to address their data challenges Our clients include 40 enterprises with over 10B in deposits AUM collectively representing about 1B customers...

  • Data Engineer-AWS, Python

    4 days ago

    Noida, Uttar Pradesh, India 3Pillar Global Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Embark on an exciting journey into the realm of software development with 3Pillar We extend an invitation for you to join our team and gear up for a thrilling adventure. At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Senior Data Engineer, you will hold a crucial...

  • AWS Data Engineer

    2 days ago

    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced AWS Data Engineer with strong expertise in Streaming Data and Big Data Technologies, AWS, Python, & Spark. The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks. Key Responsibilities: - Architect and implement...

  • AWS Data Engineer

    9 hours ago

    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced AWS Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS, Python, & Spark. The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks. Key Responsibilities: Architect and implement...

  • AWS Data Engineer

    10 hours ago

    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced AWS Data Engineer with strong expertise in Streaming Data and Big Data Technologies, AWS, Python, & Spark. The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks.Key Responsibilities:- Architect and implement...

  • AWS Data Engineer

    39 minutes ago

    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced AWS Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS, Python, & Spark. The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks. Key Responsibilities: Architect and implement...

  • AWS Data Engineer

    2 days ago

    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced AWS Data Engineer with strong expertise in Streaming Data and Big Data Technologies, AWS, Python, & Spark. The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks.Key Responsibilities:Architect and implement...