Airflow

2 weeks ago

Andhra Pradesh, India Virtusa Full time

Mandatory Skills: AWS Data Services, Python, SQL, Data Lake

Secondary Skills: DBT, Airflow, DevOps

JD: We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team.

Key Responsibilities:
Design, develop, and maintain scalable data pipelines and systems on AWS.

Implement and manage ETL processes using Python and Apache Airflow.

Develop and execute comprehensive testing plan and test cases to ensure data quality and integrity.

Collaborate with cross-functional teams to understand data requirements and deliver solutions.

Optimize and troubleshoot data workflows and performance issues.

Document and maintain data architecture, processes, and systems.

Experience in operating Scrumban or POD delivery model.

**Required Skills**:
Proficiency in AWS services such as S3, Redshift, Lambda, and Glue.

Strong programming skills in Python for data manipulation and automation.

Experience with Apache Airflow for orchestrating complex workflows.

Solid understanding of testing methodologies and tools or python scripting for data validation.

Knowledge of data integration tools such as Talend, Nucleus, or similar.

Knowledge of data transformation tools such as Data Build Tool or Abinito will be an Advantage.

Preferred Qualifications:
Bachelors or Masters degree in Computer Science, Engineering, or a related field.

7+ years of experience in data engineering or a related role.

Experience with big data technologies such as Hadoop, Spark, or Kafka.

Strong problem-solving skills and attention to detail.

Excellent communication and collaboration skills.

AWS Certification

**About Virtusa**

Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 30,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.

Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.

Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.

  • Python (Airflow/DBT) Engineer

    3 days ago

    uttar pradesh, India Luxoft Full time

    Project Description:This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure. The goal is to enhance data accessibility, performance, and governance while enabling advanced analytics and automation.The resource will join an ongoing effort to accelerate...

  • Python (Airflow/DBT) Engineer

    2 days ago

    uttar pradesh, India Luxoft Full time

    Project Description: This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure. The goal is to enhance data accessibility, performance, and governance while enabling advanced analytics and automation. The resource will join an ongoing effort to accelerate...

  • Python Pyspark

    6 days ago

    Andhra Pradesh, India Virtusa Full time

    Data Engineer Position Summary Minimum Qualifications 6 Years overall IT experience with minimum 4 years of work experience in below tech skills Tech Skills Proficient in Python scripting and PySpark for data processing tasks Strong SQL capabilities with hands on experience managing big data using ETL tools like Informatica Experience with the AWS cloud...

  • Mlops

    2 weeks ago

    Andhra Pradesh, India Virtusa Full time

    Must have 7+ years of total IT experience in Data engineering and 3 years as MLOps engineer. Designing, developing, and implementing robust MLOps pipelines using industry-leading tools like MLflow, Apache Airflow etc. Working Experience with AWS and Databricks. Must have strong Proficiency in Python, Pyspark, SQL, Machine Learning, NLP, Deep Learning,...

  • Databricks + Pyspark

    4 days ago

    Andhra Pradesh, India Virtusa Full time

    **Detailed Job Description for**Databricks + PySpark Developer**: - Data Pipeline Development: Design, implement, and maintain scalable and efficient data pipelines using PySpark and Databricks for ETL processing of large volumes of data. - Cloud Integration: Develop solutions leveraging Databricks on cloud platforms (AWS/Azure/GCP) to process and analyze...

  • Data Engineering Lead Analyst

    2 weeks ago

    Hyderabad, Telangana Andhra Pradesh, India The Cigna Group Full time

    Data Engineering Lead Analyst Position Overview Evernorth Core Platform Engineering team is looking for a Data Engineer to design develop and implement robust data pipelines for Packaged Business Capabilities PBCs You ll work closely with data scientists analysts and other engineers to ensure high data quality performance and availability The ideal candidate...

  • Application Development Advisor

    6 days ago

    Andhra Pradesh, India The Cigna Group Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Role Title:Application Developer Advisor –Full Stack Data EngineerPosition Summary:Cigna, a leading Health Services company, is looking for an exceptional engineer. The Full Stack Data Engineer is responsible for the delivery of a business need starting from understanding the requirements to deploying the software into production. This role requires you to...

  • Automation Engineer Advisor

    5 days ago

    Andhra Pradesh, India The Cigna Group Full time ₹ 60,00,000 - ₹ 1,20,00,000 per year

    Automation Engineer AdvisorPosition OverviewThe Automation Engineer advisor will be responsible for the strategy and implementation of designing and architecting automation frameworks for backend testing and data analysis of applications for Cigna's Individual and Family plans.We are looking for a skilled automation test lead with strong ETL Knowledge who...

  • Data Scientist at Hyderabad

    2 days ago

    Hyderabad, Andhra Pradesh, India Brillio Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    About Brillio: Brillio is one of the fastest growing digital technology service providers and a partner of choice for many Fortune 1000 companies seeking to turn disruption into a competitive advantage through innovative digital adoption. Brillio, renowned for its world-class professionals, referred to as "Brillians", distinguishes itself through their...

  • Data Engineer

    2 days ago

    uttar pradesh, India Credgenics Full time

    About Credgenics:Credgenics is India’s first of its kind NPA resolution platform backed by credible investors including Accel Partners and Titan Capital. We work with financial institutions, Banks, NBFCs & Digital lending firms to improve the efficiency of their collection using technology, automation intelligence and optimal legal routes to facilitate the...