Senior Data Engineer

3 weeks ago

Bengaluru Karnataka India, Karnataka Sonata Software Full time

Mandatory

* Experience building data pipelines

* Highly proficient in python, Data frames

* Experience with AWS Lambda, Events (SQS), API calls, Kinesis, DynamoDB

* Highly proficient in SQL Query

* Snowflake or experience with similar analytics database

* Experience extracting data from SQL and NoSQL databases.

* Experience with CI/CD pipeline to release the code in different environment



Nice to have

* AWS CDK

* Experience with upgrading Python version and all the linked connector in AWS environment using CDK

* Experience with other modern ETL platforms

* Snowpipe - to move data from S3 bucket to Snowflake



Job Description


A Typical Day


The Senior Software Engineer supports functions which require automation/systems development, including initial development and ongoing support of the suite of applications that would help manage loan, pricing, enterprise data pipeline, or other functions depending on the department. As the Application Developer, you must be proficient in Python, Data structures and ETL skill sets that support the IT development path.


The Senior Application Developer will:

· Data Integration: Design, develop, and maintain integrations between pennymac loan origination systems, using AWS Fargate, Lambda, Kinesis, SQS, Snowpipe etc.

· Data Quality: Implement data quality rules, matching algorithms, and cleansing routines to ensure the accuracy and completeness of loan production data

· Performance Optimization: Monitor and optimize the performance of all data pipelines and their integrations to ensure efficient data processing and access, especially in relation to real-time analytic consumers.

· Collaboration: Work closely with data engineers, architects, business stakeholders, and teams managing or consuming production data to gather requirements, design solutions, and troubleshoot production incidents and issues.

· Business Continuity: Remain available to help diagnose and troubleshoot issues. On-call shifts rotate on a regular basis.


What You’ll Bring

· Bachelor’s degree or equivalent work experience

· 5+ years of relevant work experience

· Knowledge of Agile Methodologies, such as Scrum is helpful

· Experience with relational databases (Snowflake, SQL Server)

· Experience with NoSQL databases (DynamoDb)

. Experience with API calls

. Experience with software upgrade like Python, Py odbc, snowflake connector etc

· Experience designing and implementing with Data warehouse solutions leveraging Snowflake.

· Knowledge of design patterns, refactoring and unit testing

· Financial services experience, especially related to mortgages and mortgage backed securities, is desirable

  • Senior Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    Role**: Senior Data EngineerRequired Technical Skill Set: Senior Data EngineerDesired Experience Range: 7 - 10 yrsNotice Period: Immediate to 90Days onlyLocation of Requirement: BangaloreWe are currently planning to do a Virtual Interview Job Description:Python, Scala/Java, Spark, SQL, Snowflake, Airflow, Databricks, AWS Big data technologiesDesign, develop,...

  • Senior Data Engineer cum Data Scientist

    1 week ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About the RoleAs a Senior Data Engineer cum Data Scientist, you will be a key contributor to designing, developing, and operating advanced data architectures, pipelines, and analytics solutions within the Palantir ecosystemYou will leverage your deep expertise in Palantir Foundry and Gotham platforms, combined with strong AI capabilities, to solve complex...

  • Senior Azure Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Global Delivery Services Ltd Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    ERP Senior Specialist Job Title: Azure Senior Data Engineer We are looking a highly skilled and experienced Senior Azure Data Engineer to join our team. This role requires deep expertise in Azure Data Factory and Databricks, including CI/CD setup for Databricks, and strong proficiency in ETL, SQL, and modern data engineering practices. You will be the...

  • Data Engineer Senior Consultant

    7 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 1,20,000 - ₹ 3,00,000 per year

    Req ID: 341131NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer Senior Consultant to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Location: | Experience:...

  • Senior GCP Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India, Karnataka Ascendion Full time

    Job Title: Senior GCP Data Engineer (7 - 12 Years)Job Type: Full-TimeWork Mode: HybridLocations: Bengaluru, Hyderabad, Chennai, PuneJob Summary:We are looking for a talented GCP Big Query Data Engineer with strong SQL skills and basic proficiency in Python to join our data engineering team. The ideal candidate should have hands-on experience working with...

  • Senior Data Scientist

    1 week ago

    Bengaluru, Karnataka, India Enable Data Incorporated Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is looking for a talented and experienced Senior Data Scientist to join our innovative team. In this role, you will be responsible for driving data science initiatives, developing predictive models, and providing insights that guide key business decisions. The ideal candidate has a strong background in statistical analysis, machine...

  • Senior Data Scientist

    2 weeks ago

    Bengaluru, Karnataka, India Enable Data Incorporated Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is looking for a talented and experienced Senior Data Scientist to join our innovative team. In this role, you will be responsible for driving data science initiatives, developing predictive models, and providing insights that guide key business decisions. The ideal candidate has a strong background in statistical analysis, machine...

  • Senior Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India, Karnataka ValueLabs Full time

    ValueLabs is a global technology consulting and services company, driven by innovation and excellence. With operations in over 30 countries and a client-first culture, we deliver cutting-edge digital solutions across industries. Our teams are empowered to solve complex challenges, build secure systems, and drive transformation at scale. Join us and be part...

  • Senior Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    Greetings from TCS!Job Title: Senior Data EngineeerRequired Skillset: Azure Data Factory, Databricks, Delta Live Tables, Python, SQL, Terraform, YAML, Azure Services, Data Warehousing, CI/CD, Data Governance, Purview, Informatica.Location: KolkataExperience Range: 8+yrs Job DescriptionMust-Have** Strong proficiency in Azure Data Factory, Databricks, Delta...

  • Data Entry Operator

    1 day ago

    Bengaluru, Karnataka, India Design Engineer Full time ₹ 2,64,000 per year

    We are seeking a dedicated and detail-oriented Data Entry Operator (DEO) to support R&D projects. The role involves accurate data entry, documentation, and maintenance of research records in secure systems. The DEO will assist scientists, engineers, and administrative staff by ensuring timely and error-free handling of project information.Key...