Sr. Data Engineer
2 weeks ago
***W2 or Self Inc. Only | this is not open to subcontracting (no C2C)***
Senior Data Engineer to help the design/build of the enterprise data platformsDesign, Develop and Implement
- End-to-end data solutions (ingest, storage, integration, processing, access) on AWS.
- Data intake/ request/ onboarding services and service documentation.
- Data ingestion services for batch/ real time data ingest and service documentation.
- Data processing services for batch/ real time (Glue/ Kinesis/ EMR) and service documentation.
- Data storage services for data lake (S3)/ data warehouses (RDS/ Redshift)/ data marts and service documentation.
- Data services layer including Athena, Redshift, RDS, microservices and APIs.
- Pipeline orchestration services including lambda, step functions, MWAA (optional).
- Data security services (IAM/ KMS/ SM/ encryption/ anonymization/ RBAC) and service documentation.
- Data access provisioning services (Accounts, IAM Roles RBAC), processes, documentation and education.
- Data provisioning services for data consumption patterns including microservices, APIs and extracts.
- Metadata capture and catalog services for data lake (S3/ Athena), data warehouses (RDS/ Redshift), Microservices/ APIs.
- Metadata capture and catalog services for pipeline/ log data for monitoring / support.
- Architect and implement CI/ CD strategy for EDP .
- Implement high velocity streaming solutions using Amazon Kinesis, SQS, and SMS.
- Migrate data from traditional relational database systems, file systems, NAS shares to AWS relational databases such as Amazon RDS, Aurora, and Redshift.
- Migrate data from APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift.
- Implement cost/ spend monitoring for AWS based data platform.
- Implement audit reporting for access of AWS based data platform.
- Contribute to the implementation of a data strategy to enable a robust, holistic view of data - driven decision making.
- Partner with immediate engineering team, product owner, IT, partners on EDP agenda.
- Leverage and continuously develop best practices, standards, and frameworks.
- Provide technology thought leadership, consulting, and coaching/ mentoring.
- Work with scrum master to develop and own backlog, stories, epics, sprints.
- System design and Architecture for products/ applications for EDP.
- Work closely with various stakeholders (BI, AIML, MDM and other teams) to understand their use-cases and design optimal solution for them.
- Execute technical design and infrastructure strategy for EDP .
- Implement POCs on any new technology or tools to be implemented on EDP and onboard for real use-case.
Qualifications
- Bachelor's degree in computer science, Software Engineering, MIS or equivalent combination of education and experience.
- Experience implementing, supporting data platforms on AWS for large enterprises.
- Full stack development experience building secure internal facing applications.
- Programming experience with Java, Python/ Scala, Shell scripting.
- Solid experience of AWS services such as CloudFormation, S3, Glue, EMR/ Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, SM etc.
- Solid experience implementing solutions on AWS based data lakes preferred.
- Experience implementing metadata solutions leveraging AWS non-relational data solutions such as ElastiCache and DynamoDB.
- AWS Solutions Architect or AWS Big Data Certification preferred.
- Experience in AWS data lake, data warehouse and business analytics.
- Experience and understanding of various core AWS services such as IAM, Cloud Formation, EC2, S3, EMR/Spark, Glue, Datasync, CloudHealth, CloudWatch, Lambda, Athena, and Redshift.
- Experience in system analysis, design, development, and implementation of data ingestion pipeline in AWS.
- Experience with DevOps and Continuous Integration/ Delivery (CI/ CD) concepts and tools.
- Experience with business intelligence tools such as Tableau, Power BI or equivalent.
- Knowledge of ETL/ ELT.
- Awareness of Data Management & Governance tools.
- Working experience with Hadoop, HDFS, SQOOP, Hive, Python, and Spark is desired.
- Experience working on Agile projects.
Requirements
data engineering, AWS, data lake / data warehouses
-
Sr. Azure Data Engineer
1 week ago
india SolarSys Innovations Private Ltd Full timeCompany Description SolarSys Innovations is a leading software company that specializes in helping organizations undergo digital transformations. Our goal is to differentiate our clients from their competitors and enhance their engagement with customers, partners, and employees. We understand that each customer landscape is unique and requires tailored...
-
Sr Specialist, Data Solutions
3 hours ago
india Liberty Data Analytics Full timeSenior Specialist, Data Solutions Provides, builds, and maintains data solutions to meet the needs of the enterprise. Helps ensure those solutions can be scaled appropriately. Under minimal to no direction, works within broad limits and authority on highly complex assignments requiring specialized knowledge. Viewed as senior resource in given field for...
-
Sr. Engineer
1 week ago
india QRC Group, LLC Full timeJob Description Sr Engineering with experience in medical device or pharma industries, and packaging areas. Experience in Parenteral Inspection and/or Packaging equipmentKnowledge in Engineering ProjectsKnowledge in CAPA, Change Management and Compliance Shift: 2ND and 3rd shifts Requirements Doctorate or Master's degree and 3 years of Engineering...
-
Sr Engineer
2 weeks ago
india QRC Group, LLC Full timeJob Description Sr Engineer with experience in regulated industry (Medical Device or Pharma) with knowledge in: Packaging equipment.Design of new equipment process.Engineering ProjectValidations Requirements Doctorate degree or Master’s degree and 3 years of Engineering experience orBachelor’s degree in Engineering and 5 years of Engineering experience...
-
Sr Software Engineer
4 weeks ago
india Deltek Full time16-Apr-2024 Sr Software Engineer India-Remote 9263BR Company Summary As the recognized global standard for project-based businesses, Deltek delivers software and information solutions to help organizations achieve their purpose. Our market leadership stems from the work of our diverse employees who are united by a passion for learning,...
-
Sr Engineer
1 week ago
india QRC Group, LLC Full timeJob Description Sr Engineer with experience in medical devices or pharma industry.Experience and knowledge in: Packing Equipment Design of New Equipment ProcessEngineering Project Validations and Commissioning CAPA and Compliance Availability for 1st or 2nd shift Requirements Doctorate degree or Master's degree and 3 years of Engineering experience...
-
Data Engineer
1 month ago
india INSIGHT GLOBAL SOLUTIONS Full timeData Engineer Must Haves :- Bachelors Degree in science, technology, or another related field.- 3+ years of data engineering experience.- 3+ years of experience with Cloud platforms like GCP, Azure, Airflow.- 4+ years of data warehouse, relational database, and SQL experience.- 4+ years of supporting ETL process and job scheduling experience.- 3+ years of...
-
Sr Mgr, Engineering
2 weeks ago
india Deltek Full time07-Mar-2024 Sr Manager Engineering India-Remote 9183BR Company Summary As the recognized global standard for project-based businesses, Deltek delivers software and information solutions to help organizations achieve their purpose. Our market leadership stems from the work of our diverse employees who are united by a passion for...
-
Principal Sr/Engineer
2 weeks ago
india Photon Full timeJob description:- The Principal/Sr engineer, reporting to the Sr Manager, Flagship Mobile Apps, will work on the team that brings CVS customers their “Super app” – the customer-facing Flagship CVS Shopping and Pharmacy apps. As a Principal/Sr Engineer, you will be helping advance our architecture and Mobile Product, working with...
-
Senior Machine Learning Engineer
2 weeks ago
india K2 Data & Analytics Full timeJob Description : We are on the lookout for an enthusiastic ML Engineer with over five years of experience to be part of our team, which is focused on developing cutting-edge Video Ad Data technologies. The perfect fit for this role would be someone with a solid grasp of machine learning algorithms and a background in designing and executing large-scale...
-
Principal Data Engineer
4 weeks ago
india Cornerstone OnDemand Full timeWe're looking for a Principal Data Engineer This role is Remote Principal Data Engineer We are seeking a talented Principal Data Engineer in Pune, India. Reporting to the Sr. Manager – Data Engineering. The right candidate has strong communication skills, passion for solving business problems with data, domain knowledge in Finance –...
-
Sr Data Consultant/Architect
3 weeks ago
india NEXUS CORPORATION Full timeJob Description We are seeking a Sr Data Consultant/Architect to join our Japan team in Tokyo. As a Sr Data Consultant you will be accountable for developing data products with decentralized data architecture. You will be able to apply your data warehousing experience as well as learn modern and efficient ways of delivering data to business partners. You...
-
Senior Azure Data Engineer
3 weeks ago
india YASH Technologies Full timeSr. Software Engineer - Azure Experience: 4 - 7 years of Experience What you should have? Azure Data Factory, Azure DevOps CI/CD, Azure Data Lake, Azure Synapse, Azure Databricks, PySpark, GitHub. Need 4+ Years of experience Good analytical skills with excellent knowledge of SQL. Well versed with Azure Services Must have experience and knowledge on...
-
Aptus Data Labs
15 hours ago
india Aptus Data Labs Full timeRole - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...
-
Sr. Fullstack Engineer
4 weeks ago
India SolutionSavvy Full timeFull stack DeveloperJr. Developer - Salary Range 2.5 LPA - 4 LPA (negotiable)Mid Sr. Developer - Salary Range 4 LPA- 10 LPA (negotiable)Sr. Developer - Salary Range 10LPA - 15 LPA (negotiable)Architect Level - 15 LPA - 20 LPAWe are a start-up building custom products for our clients. Currently, we are working with German clients to build an end-end product...
-
Sr. Fullstack Engineer
4 weeks ago
india SolutionSavvy Full timeFull stack Developer Jr. Developer - Salary Range 2.5 LPA - 4 LPA (negotiable) Mid Sr. Developer - Salary Range 4 LPA- 10 LPA (negotiable) Sr. Developer - Salary Range 10LPA - 15 LPA (negotiable) Architect Level - 15 LPA - 20 LPA We are a start-up building custom products for our clients. Currently, we are working with German clients to build an end-end...
-
Sr. Software Engineer II
4 weeks ago
india Bottomline Full timeRole We are looking for a Sr. Software Engineer II to innovate, win and grow with us in Bangalore! The Sr. Software Engineer II is involved in all stages of development including planning, implementation, testing, documentation, release and product support; work to debug and implement new changes to systems as they are prioritized. He/she must have good...
-
Senior Data Engineer
1 week ago
india ParentPay Group - India Full timeJD – Sr. Data Engineer SQL performance SQL query tuning SQL monitoring SQL Data warehousing SQL Indexing Star Schema Design. ETL processes Data modelling Azure Data Factory Azure Storage Accounts Blob Storage MUST HAVE SKILLS: - STAR SCHEMA DESIGN, SQL QUERY TUNNING, ETL PROCESSES, AZURE. About Organization:- ParentPay Group is a Europe’s leading...
-
MLOps Engineer
15 hours ago
india GOOGNU DATA SERVICES PRIVATE LIMITED Full timeJob Description :- Design and Implement MLOps Infrastructure: Build and maintain robust MLOps workflows using Azure Machine Learning and other Azure services to automate machine learning lifecycle from data preparation to model deployment and monitoring.- Collaborate with Cross-Functional Teams: Work closely with data scientists, software developers, and IT...
-
Sr Backend Engineer
1 month ago
india Egnyte Full timeDescription Sr Backend Engineer EGNYTE YOUR CAREER. SPARK YOUR PASSION. Egnyte is a place where we spark opportunities for amazing people. We believe that every role has meaning, and every Egnyter should be respected. With over 22,000 customers worldwide and growing, you can make an impact by protecting their valuable data. When...