Associate Iii

Role Proficiency This role requires proficiency in data pipeline development including coding and testing data pipelines for ingesting wrangling transforming and joining data from various sources Must be adept at using ETL tools such as Informatica Glue Databricks and DataProc with coding skills in Python PySpark and SQL Works independently and demonstrates proficiency in at least one domain related to data with a solid understanding of SCD concepts and data warehousing principles Outcomes Collaborate closely with data analysts data scientists and other stakeholders to ensure data accessibility quality and security across various data sources rnDesign develop and maintain data pipelines that collect process and transform large volumes of data from various sources Implement ETL Extract Transform Load processes to facilitate efficient data movement and transformation Integrate data from multiple sources including databases APIs cloud services and third-party data providers Establish data quality checks and validation procedures to ensure data accuracy completeness and consistency Develop and manage data storage solutions including relational databases NoSQL databases and data lakes Stay updated on the latest trends and best practices in data engineering cloud technologies and big data tools Measures of Outcomes Adherence to engineering processes and standards Adherence to schedule timelines Adhere to SLAs where applicable of defects post delivery of non-compliance issues Reduction of reoccurrence of known defects Quickly turnaround production bugs Completion of applicable technical domain certifications Completion of all mandatory training requirementst Efficiency improvements in data pipelines e g reduced resource consumption faster run times Average time to detect respond to and resolve pipeline failures or data issues Outputs Expected Code Development Develop data processing code independently ensuring it meets performance and scalability requirements Documentation Create documentation for personal work and review deliverable documents including source-target mappings test cases and results Configuration Follow configuration processes diligently Testing Create and conduct unit tests for data pipelines and transformations to ensure data quality and correctness Validate the accuracy and performance of data processes Domain Relevance Develop features and components with a solid understanding of the business problems being addressed for the client Understand data schemas in relation to domain-specific contexts such as EDI formats Defect Management Raise fix and retest defects in accordance with project standards Estimation Estimate time effort and resource dependencies for personal work Knowledge Management Consume and contribute to project-related documents SharePoint libraries and client universities Design Understanding Understand design and low-level design LLD and link it to requirements and user stories Certifications Obtain relevant technology certifications to enhance skills and knowledge Skill Examples Proficiency in SQL Python or other programming languages utilized for data manipulation Experience with ETL tools such as Apache Airflow Talend Informatica AWS Glue Dataproc and Azure ADF Hands-on experience with cloud platforms like AWS Azure or Google Cloud particularly with data-related services e g AWS Glue BigQuery Conduct tests on data pipelines and evaluate results against data quality and performance specifications Experience in performance tuning data processes Proficiency in querying data warehouses Knowledge Examples Knowledge Examples Knowledge of various ETL services provided by cloud providers including Apache PySpark AWS Glue GCP DataProc DataFlow and Azure ADF ADLF Understanding of data warehousing principles and practices Proficiency in SQL for analytics including windowing functions Familiarity with data schemas and models Understanding of domain-related data and its implications Additional Comments Job Title Junior Data Engineer Experience 2-4 years Employment Type Full-time Role Summary We are looking for a Junior Data Engineer who is passionate about building scalable data pipelines and working with modern data technologies The ideal candidate should have hands-on experience in ETL development data integration and cloud-based data solutions preferably on AWS You will work closely with senior data engineers data architects and analysts to develop and maintain data pipelines ensuring data is accurate reliable and available for business and analytics needs This is an excellent opportunity to grow into a data engineering expert in a modern cloud ecosystem Key Responsibilities o Design develop and maintain ETL ELT pipelines for structured and unstructured data sources o Work with senior engineers to implement data lake and data warehouse solutions using AWS services such as S3 Glue Redshift and Lambda o Develop and optimize Spark Databricks jobs for data transformation and processing o Write efficient Python or SQL scripts for data cleaning transformation and validation o Collaborate with data architects to implement data models and data quality frameworks o Participate in code reviews testing and deployment processes to maintain code quality and stability o Troubleshoot and resolve data-related issues in development and production environments o Document technical processes data flows and pipeline logic for maintainability and transparency o Learn and contribute to continuous improvements in automation performance and scalability of data systems Technical Skills and Tools Mandatory o Good understanding of ETL concepts data warehousing and data integration techniques o Hands-on experience with Python for data manipulation and scripting o Working knowledge of SQL and experience writing optimized queries o Familiarity with AWS data services like S3 Glue Redshift Athena or equivalent cloud tools Azure Data Factory GCP BigQuery o Basic exposure to Apache Spark or Databricks for distributed data processing o Version control and collaboration tools Git GitHub Bitbucket Good-to-Have o Exposure to Airflow or any job orchestration tool o Knowledge of data modelling and dimensional design principles o Understanding of data quality checks logging and monitoring o Familiarity with DevOps CI-CD pipelines o Awareness of streaming data concepts Kafka Kinesis Soft Skills Attributes o Strong analytical and problem-solving skills with attention to detail o Willingness to learn and adapt quickly in a fast-paced data-driven environment o Good communication and collaboration skills to work effectively within cross-functional teams o A proactive attitude toward automation documentation and best practices o Curiosity to explore emerging tools and technologies in the data ecosystem Education Experience o Bachelor s degree in Computer Science Information Technology or related discipline o 2-4 years of hands-on experience in data engineering ETL development or data integration o Prior experience working in a cloud data environment AWS Azure GCP preferred Preferred Qualifications o AWS Certified Data Practitioner or Data Engineer Associate or equivalent certification on Azure GCP o Exposure to modern data stack tools such as dbt Snowflake or Delta Lake o Experience contributing to end-to-end data pipeline projects in production environments Skills ETL Data Warehousing AWS About Company UST is a global digital transformation solutions provider For more than 20 years UST has worked side by side with the world s best companies to make a real impact through transformation Powered by technology inspired by people and led by purpose UST partners with their clients from design to operation With deep domain expertise and a future-proof philosophy UST embeds innovation and agility into their clients organizations With over 30 000 employees in 30 countries UST builds for boundless impact touching billions of lives in the process