Sr. Data Engineer

2 weeks ago


india MDMS Recruiting LLC Full time
Job Description

***W2 or Self Inc. Only | this is not open to subcontracting (no C2C)***

Senior Data Engineer to help the design/build of the enterprise data platforms

Design, Develop and Implement

  • End-to-end data solutions (ingest, storage, integration, processing, access) on AWS.
  • Data intake/ request/ onboarding services and service documentation.
  • Data ingestion services for batch/ real time data ingest and service documentation.
  • Data processing services for batch/ real time (Glue/ Kinesis/ EMR) and service documentation.
  • Data storage services for data lake (S3)/ data warehouses (RDS/ Redshift)/ data marts and service documentation.
  • Data services layer including Athena, Redshift, RDS, microservices and APIs.
  • Pipeline orchestration services including lambda, step functions, MWAA (optional).
  • Data security services (IAM/ KMS/ SM/ encryption/ anonymization/ RBAC) and service documentation.
  • Data access provisioning services (Accounts, IAM Roles RBAC), processes, documentation and education.
  • Data provisioning services for data consumption patterns including microservices, APIs and extracts.
  • Metadata capture and catalog services for data lake (S3/ Athena), data warehouses (RDS/ Redshift), Microservices/ APIs.
  • Metadata capture and catalog services for pipeline/ log data for monitoring / support.
  • Architect and implement CI/ CD strategy for EDP .
  • Implement high velocity streaming solutions using Amazon Kinesis, SQS, and SMS.
  • Migrate data from traditional relational database systems, file systems, NAS shares to AWS relational databases such as Amazon RDS, Aurora, and Redshift.
  • Migrate data from APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift.
  • Implement cost/ spend monitoring for AWS based data platform.
  • Implement audit reporting for access of AWS based data platform.
  • Contribute to the implementation of a data strategy to enable a robust, holistic view of data - driven decision making.
  • Partner with immediate engineering team, product owner, IT, partners on EDP agenda.
  • Leverage and continuously develop best practices, standards, and frameworks.
  • Provide technology thought leadership, consulting, and coaching/ mentoring.
  • Work with scrum master to develop and own backlog, stories, epics, sprints.
  • System design and Architecture for products/ applications for EDP.
  • Work closely with various stakeholders (BI, AIML, MDM and other teams) to understand their use-cases and design optimal solution for them.
  • Execute technical design and infrastructure strategy for EDP .
  • Implement POCs on any new technology or tools to be implemented on EDP and onboard for real use-case.

Qualifications  

  • Bachelor's degree in computer science, Software Engineering, MIS or equivalent combination of education and experience.
  • Experience implementing, supporting data platforms on AWS for large enterprises.
  • Full stack development experience building secure internal facing applications.
  • Programming experience with Java, Python/ Scala, Shell scripting.
  • Solid experience of AWS services such as CloudFormation, S3, Glue, EMR/ Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, SM etc.
  • Solid experience implementing solutions on AWS based data lakes preferred.
  • Experience implementing metadata solutions leveraging AWS non-relational data solutions such as ElastiCache and DynamoDB.
  • AWS Solutions Architect or AWS Big Data Certification preferred.
  • Experience in AWS data lake, data warehouse and business analytics.
  • Experience and understanding of various core AWS services such as IAM, Cloud Formation, EC2, S3, EMR/Spark, Glue, Datasync, CloudHealth, CloudWatch, Lambda, Athena, and Redshift.
  • Experience in system analysis, design, development, and implementation of data ingestion pipeline in AWS.
  • Experience with DevOps and Continuous Integration/ Delivery (CI/ CD) concepts and tools.
  • Experience with business intelligence tools such as Tableau, Power BI or equivalent.
  • Knowledge of ETL/ ELT.
  • Awareness of Data Management & Governance tools.
  • Working experience with Hadoop, HDFS, SQOOP, Hive, Python, and Spark is desired.
  • Experience working on Agile projects.

 

 


Requirements
data engineering, AWS, data lake / data warehouses

  • india SolarSys Innovations Private Ltd Full time

    Company Description SolarSys Innovations is a leading software company that specializes in helping organizations undergo digital transformations. Our goal is to differentiate our clients from their competitors and enhance their engagement with customers, partners, and employees. We understand that each customer landscape is unique and requires tailored...


  • india Liberty Data Analytics Full time

    Senior Specialist, Data Solutions Provides, builds, and maintains data solutions to meet the needs of the enterprise. Helps ensure those solutions can be scaled appropriately. Under minimal to no direction, works within broad limits and authority on highly complex assignments requiring specialized knowledge. Viewed as senior resource in given field for...

  • Sr. Engineer

    1 week ago


    india QRC Group, LLC Full time

    Job Description Sr Engineering with experience in medical device or pharma industries, and packaging areas. Experience in Parenteral Inspection and/or Packaging equipmentKnowledge in Engineering ProjectsKnowledge in CAPA, Change Management and Compliance Shift: 2ND and 3rd shifts Requirements Doctorate or Master's degree and 3 years of Engineering...

  • Sr Engineer

    2 weeks ago


    india QRC Group, LLC Full time

    Job Description Sr Engineer with experience in regulated industry (Medical Device or Pharma) with knowledge in: Packaging equipment.Design of new equipment process.Engineering ProjectValidations Requirements Doctorate degree or Master’s degree and 3 years of Engineering experience orBachelor’s degree in Engineering and 5 years of Engineering experience...

  • Sr Software Engineer

    4 weeks ago


    india Deltek Full time

    16-Apr-2024 Sr Software Engineer India-Remote 9263BR Company Summary As the recognized global standard for project-based businesses, Deltek delivers software and information solutions to help organizations achieve their purpose. Our market leadership stems from the work of our diverse employees who are united by a passion for learning,...

  • Sr Engineer

    1 week ago


    india QRC Group, LLC Full time

    Job Description Sr Engineer with experience in medical devices or pharma industry.Experience and knowledge in:  Packing Equipment Design of New Equipment ProcessEngineering Project Validations and Commissioning CAPA and Compliance  Availability for 1st or 2nd shift Requirements Doctorate degree or Master's degree and 3 years of Engineering experience...

  • Data Engineer

    1 month ago


    india INSIGHT GLOBAL SOLUTIONS Full time

    Data Engineer Must Haves :- Bachelors Degree in science, technology, or another related field.- 3+ years of data engineering experience.- 3+ years of experience with Cloud platforms like GCP, Azure, Airflow.- 4+ years of data warehouse, relational database, and SQL experience.- 4+ years of supporting ETL process and job scheduling experience.- 3+ years of...

  • Sr Mgr, Engineering

    2 weeks ago


    india Deltek Full time

    07-Mar-2024 Sr Manager Engineering India-Remote 9183BR Company Summary As the recognized global standard for project-based businesses, Deltek delivers software and information solutions to help organizations achieve their purpose. Our market leadership stems from the work of our diverse employees who are united by a passion for...

  • Principal Sr/Engineer

    2 weeks ago


    india Photon Full time

    Job description:-  The Principal/Sr engineer, reporting to the Sr Manager, Flagship Mobile Apps, will work on the team that brings CVS customers their “Super app” – the customer-facing Flagship CVS Shopping and Pharmacy apps.  As a Principal/Sr Engineer, you will be helping advance our architecture and Mobile Product, working with...


  • india K2 Data & Analytics Full time

    Job Description : We are on the lookout for an enthusiastic ML Engineer with over five years of experience to be part of our team, which is focused on developing cutting-edge Video Ad Data technologies. The perfect fit for this role would be someone with a solid grasp of machine learning algorithms and a background in designing and executing large-scale...


  • india Cornerstone OnDemand Full time

    We're looking for a Principal Data Engineer This role is Remote Principal Data Engineer We are seeking a talented Principal Data Engineer in Pune, India. Reporting to the Sr. Manager – Data Engineering. The right candidate has strong communication skills, passion for solving business problems with data, domain knowledge in Finance –...


  • india NEXUS CORPORATION Full time

    Job Description We are seeking a Sr Data Consultant/Architect to join our Japan team in Tokyo. As a Sr Data Consultant you will be accountable for developing data products with decentralized data architecture. You will be able to apply your data warehousing experience as well as learn modern and efficient ways of delivering data to business partners. You...


  • india YASH Technologies Full time

    Sr. Software Engineer - Azure Experience: 4 - 7 years of Experience What you should have? Azure Data Factory, Azure DevOps CI/CD, Azure Data Lake, Azure Synapse, Azure Databricks, PySpark, GitHub. Need 4+ Years of experience Good analytical skills with excellent knowledge of SQL. Well versed with Azure Services Must have experience and knowledge on...

  • Aptus Data Labs

    15 hours ago


    india Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...


  • India SolutionSavvy Full time

    Full stack DeveloperJr. Developer - Salary Range 2.5 LPA - 4 LPA (negotiable)Mid Sr. Developer - Salary Range 4 LPA- 10 LPA (negotiable)Sr. Developer - Salary Range 10LPA - 15 LPA (negotiable)Architect Level - 15 LPA - 20 LPAWe are a start-up building custom products for our clients. Currently, we are working with German clients to build an end-end product...


  • india SolutionSavvy Full time

    Full stack Developer Jr. Developer - Salary Range 2.5 LPA - 4 LPA (negotiable) Mid Sr. Developer - Salary Range 4 LPA- 10 LPA (negotiable) Sr. Developer - Salary Range 10LPA - 15 LPA (negotiable) Architect Level - 15 LPA - 20 LPA We are a start-up building custom products for our clients. Currently, we are working with German clients to build an end-end...


  • india Bottomline Full time

    Role We are looking for a Sr. Software Engineer II to innovate, win and grow with us in Bangalore! The Sr. Software Engineer II is involved in all stages of development including planning, implementation, testing, documentation, release and product support; work to debug and implement new changes to systems as they are prioritized. He/she must have good...


  • india ParentPay Group - India Full time

    JD – Sr. Data Engineer SQL performance SQL query tuning SQL monitoring SQL Data warehousing SQL Indexing Star Schema Design. ETL processes Data modelling Azure Data Factory Azure Storage Accounts Blob Storage MUST HAVE SKILLS: - STAR SCHEMA DESIGN, SQL QUERY TUNNING, ETL PROCESSES, AZURE. About Organization:- ParentPay Group is a Europe’s leading...

  • MLOps Engineer

    15 hours ago


    india GOOGNU DATA SERVICES PRIVATE LIMITED Full time

    Job Description :- Design and Implement MLOps Infrastructure: Build and maintain robust MLOps workflows using Azure Machine Learning and other Azure services to automate machine learning lifecycle from data preparation to model deployment and monitoring.- Collaborate with Cross-Functional Teams: Work closely with data scientists, software developers, and IT...

  • Sr Backend Engineer

    1 month ago


    india Egnyte Full time

    Description Sr Backend Engineer   EGNYTE YOUR CAREER. SPARK YOUR PASSION.   Egnyte is a place where we spark opportunities for amazing people. We believe that every role has meaning, and every Egnyter should be respected. With over 22,000 customers worldwide and growing, you can make an impact by protecting their valuable data. When...