Senior Data Engineer

13 hours ago

Bengaluru Karnataka India, Karnataka Sonata Software Full time

Mandatory

* Experience building data pipelines

* Highly proficient in python, Data frames

* Experience with AWS Lambda, Events (SQS), API calls, Kinesis, DynamoDB

* Highly proficient in SQL Query

* Snowflake or experience with similar analytics database

* Experience extracting data from SQL and NoSQL databases.

* Experience with CI/CD pipeline to release the code in different environment



Nice to have

* AWS CDK

* Experience with upgrading Python version and all the linked connector in AWS environment using CDK

* Experience with other modern ETL platforms

* Snowpipe - to move data from S3 bucket to Snowflake



Job Description


A Typical Day


The Senior Software Engineer supports functions which require automation/systems development, including initial development and ongoing support of the suite of applications that would help manage loan, pricing, enterprise data pipeline, or other functions depending on the department. As the Application Developer, you must be proficient in Python, Data structures and ETL skill sets that support the IT development path.


The Senior Application Developer will:

· Data Integration: Design, develop, and maintain integrations between pennymac loan origination systems, using AWS Fargate, Lambda, Kinesis, SQS, Snowpipe etc.

· Data Quality: Implement data quality rules, matching algorithms, and cleansing routines to ensure the accuracy and completeness of loan production data

· Performance Optimization: Monitor and optimize the performance of all data pipelines and their integrations to ensure efficient data processing and access, especially in relation to real-time analytic consumers.

· Collaboration: Work closely with data engineers, architects, business stakeholders, and teams managing or consuming production data to gather requirements, design solutions, and troubleshoot production incidents and issues.

· Business Continuity: Remain available to help diagnose and troubleshoot issues. On-call shifts rotate on a regular basis.


What You’ll Bring

· Bachelor’s degree or equivalent work experience

· 5+ years of relevant work experience

· Knowledge of Agile Methodologies, such as Scrum is helpful

· Experience with relational databases (Snowflake, SQL Server)

· Experience with NoSQL databases (DynamoDb)

. Experience with API calls

. Experience with software upgrade like Python, Py odbc, snowflake connector etc

· Experience designing and implementing with Data warehouse solutions leveraging Snowflake.

· Knowledge of design patterns, refactoring and unit testing

· Financial services experience, especially related to mortgages and mortgage backed securities, is desirable

  • Senior Data Engineer

    13 hours ago

    Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    Role**: Senior Data EngineerRequired Technical Skill Set: Senior Data EngineerDesired Experience Range: 7 - 10 yrsNotice Period: Immediate to 90Days onlyLocation of Requirement: BangaloreWe are currently planning to do a Virtual Interview Job Description:Python, Scala/Java, Spark, SQL, Snowflake, Airflow, Databricks, AWS Big data technologiesDesign, develop,...

  • Senior Data Engineer

    1 week ago

    Bengaluru, Karnataka, India, Karnataka ITC Infotech Full time

    Senior Data Engineer - Azure StackExperience - 3-12 yearsLocation : Bangalore, Kolkata, Gurgaon, PuneWe are looking for a highly skilled Senior Data Engineer with deep expertise in Azure Stack to join our growing team. As a Senior Data Engineer, you will be instrumental in designing and developing robust data pipelines, data lakes, and cloud-based solutions...

  • Senior Data Engineer

    1 week ago

    Bengaluru, Karnataka, India, Karnataka Publicis Production Full time

    Job Summary:We are seeking a proactive and self-motivated Senior Data Engineer with a proven track record in building scalable cloud-based data solutions across multiple cloud platforms to support our work in architecting, building and maintaining the data infrastructure. The specific focus for this role will start with GCP however we require experience with...

  • Senior Data Engineer

    1 week ago

    Bengaluru, Karnataka, India, Karnataka 6thStreet.com Full time

    Job Title: Senior Data Engineer Location: Bangalore Experience: 3-5 Years Department: Data & Analytics Employment Type: Full-timeAbout the RoleWe’re looking for a Senior Data Engineer to lead the design, development, and scaling of our data infrastructure. You’ll shape best practices, mentor junior engineers, and ensure our data ecosystem empowers...

  • Senior Data Engineer

    1 week ago

    Bengaluru, Karnataka, India, Karnataka JRD Systems Full time

    We are seeking a Senior Data Engineer with deep expertise in AWS Cloud Development. This role is critical to both the development of new data products and the modernization of existing platforms.The ideal candidate is a seasoned data engineer with hands-on experience designing, building, and optimizing large-scale data pipelines and architectures in both...

  • Senior Data Engineer

    1 week ago

    Bengaluru, Karnataka, India, Karnataka Avathon Full time

    Who We Are & Why Join UsAvathon is revolutionizing industrial AI with a powerful platform that enables businesses to harness the full potential of their operational data. Our technology seamlessly integrates and contextualizes siloed datasets, providing a 360-degree operational view that enhances decision-making and efficiency. With advanced capabilities...

  • Senior Data Engineer cum Data Scientist

    1 week ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About the RoleAs a Senior Data Engineer cum Data Scientist, you will be a key contributor to designing, developing, and operating advanced data architectures, pipelines, and analytics solutions within the Palantir ecosystemYou will leverage your deep expertise in Palantir Foundry and Gotham platforms, combined with strong AI capabilities, to solve complex...

  • Senior Data Engineer

    1 week ago

    Bengaluru, Karnataka, India, Karnataka Grid Dynamics Full time

    About Company Grid Dynamics (Nasdaq:GDYN) is a digital-native technology services provider that accelerates growth and bolsters competitive advantage for Fortune 1000 companies. Grid Dynamics provides digital transformation consulting and implementation services in omnichannel customer experience, big data analytics, search, artificial intelligence, cloud...

  • Senior Data Engineer

    1 week ago

    Bengaluru, Karnataka, India, Karnataka Benchire Full time

    Hi, We are Hiring for a Senior data Engineer with 6-8 years of Experience for a IT Services company based in Bangalore .Requirements:· Min 6-8 years of professional experience· Minimum of three years of professional experience with Python· Professional experience in the field of data (e.g., data science, data engineering, data warehousing)· Experience...

  • Senior Azure Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Global Delivery Services Ltd Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    ERP Senior Specialist Job Title: Azure Senior Data Engineer We are looking a highly skilled and experienced Senior Azure Data Engineer to join our team. This role requires deep expertise in Azure Data Factory and Databricks, including CI/CD setup for Databricks, and strong proficiency in ETL, SQL, and modern data engineering practices. You will be the...