Senior QA Engineer (Software & AI Quality)

About FPT Smart Cloud

FPT Smart Cloud (FCI), a member of FPT Corporation, is a pioneer in providing AI and Cloud solutions in Vietnam. FCI was founded with the mission of driving significant improvements in business productivity and operational agility.

FPT Smart Cloud aims to become an industry leader by building a strong technological foundation, developing a diverse ecosystem of products, and expanding global connectivity.

About the Team

You will join a product team developing AI-powered B2B SaaS platforms for enterprise customers. The team follows Agile/Scrum methodologies and works closely with Product Owners, Designers, Software Engineers, AI Engineers, and QA Engineers. AI tools are also integrated into our daily workflows.

As a Senior QA Engineer, you will take ownership of both software quality and AI behavior. Approximately 50% of your time will focus on software testing and test automation, while the remaining 50% will be dedicated to evaluating and improving AI quality.

Key Responsibilities

Software Quality (Approximately 50%)

• Define test strategies, test plans, and quality gates for each sprint and release.
• Review User Stories and Acceptance Criteria with BAs and POs before each sprint, identifying gaps and potential risks early.
• Design comprehensive test cases based on Acceptance Criteria, covering user roles, permissions, and multi-tenant data isolation.
• Test web applications, REST APIs, real-time features (WebSocket), and third-party integrations.
• Build and maintain automated UI and API regression test suites using Playwright or similar tools, integrating them into CI/CD pipelines.
• Leverage AI tools to accelerate and optimize test case development.
• Perform data migration testing and collaborate with DevOps engineers on basic performance testing.
• Conduct UAT with customers and monitor defects that escape into the UAT phase.

AI Quality (Approximately 50%)

• Build and maintain evaluation datasets based on real-world use cases for LLM-powered features and AI agents.
• Define evaluation criteria and scoring rubrics in collaboration with AI Engineers and BAs, covering accuracy, groundedness, hallucination, safety, and consistency.
• Conduct regression evaluations whenever prompts, models, or knowledge sources are updated. Compare different versions and identify potential quality risks.
• Perform AI safety testing, including prompt injection, personal data leakage, and unauthorized actions beyond defined policies.
• Calibrate automated LLM-as-a-Judge scores against human evaluation results.
• Monitor AI quality in production through logs, reports, and customer feedback.
• Lead root cause analysis for AI quality incidents and recommend improvement actions.

Leadership & Process Improvement

• Mentor and provide technical guidance to QA team members.
• Establish QA processes, quality metrics, and reporting standards for the team.

What Success Looks Like (First 6 Months)

• Automated regression tests are executed for every release, covering the core workflows of the squad.
• Every AI feature has a defined evaluation dataset, scoring rubric, and pass threshold agreed upon with the Product Owner.
• The number of defects escaping into UAT decreases with each release.

Requirements

Must-have Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Data Science, AI, or a related field.
• 4+ years of experience in software testing/QA, including at least 2 years working with web-based or SaaS products.
• Strong knowledge of test design techniques, including equivalence partitioning, boundary value analysis, decision tables, and state transition testing.
• Ability to produce clear and structured test documentation, including test plans, test cases, and bug reports.
• Hands-on experience with API testing tools such as Postman, reading JSON, and using SQL for data validation.
• Practical experience developing automated tests using Playwright, Cypress, Selenium, or similar frameworks, with JavaScript/TypeScript or Python, and integrating automated tests into CI pipelines.
• Hands-on experience testing or evaluating chatbots, LLMs, or other AI-powered features, including building test datasets and identifying hallucinations, edge cases, and conversation breakdowns.
• Experience working in Agile/Scrum teams and using Jira.
• Strong attention to detail and the ability to evaluate AI behavior from both business and technical perspectives.
• Ability to read and understand technical documentation in English.

Nice-to-have Qualifications

• Experience with Conversational AI, chatbot/voicebot, or CRM products.
• Familiarity with LLM evaluation tools such as Promptfoo, DeepEval, Ragas, or LangSmith.
• Experience testing real-time systems, multi-tenant platforms, or data migration processes.
• Basic knowledge of security testing, including OWASP Top 10 and authorization bypass testing, as well as performance testing tools such as k6 or JMeter.
• Experience using Python or log analysis tools for quality data analysis.
• ISTQB certification (Foundation or Advanced).

Benefits

• Competitive salary based on qualifications and experience, negotiable during the interview process.
• Social and health insurance in accordance with Vietnamese labor laws.
• A creative, open-minded, and inclusive working environment that values individuality.
• FPT Premium Care healthcare package.
• Various cultural activities and engagement programs across FCI and FPT Corporation.
• Educational support programs for children of FCI trade union members.
• Sponsorship for relevant professional courses and certifications.

Working Environment

• Location: 22nd Floor, Vinacomin Tower, No. 3 Duong Dinh Nghe Street, Cau Giay District, Hanoi.
• Working Hours: 8:30 AM – 12:00 PM and 1:00 PM – 5:30 PM.
• Working Days: Monday to Friday (Weekends off).

Contact Information

Pham Thi Ha My (Ms.) – Talent Acquisition Team Leader

• Email: [email protected]
• Phone: 0962 456 194

FPT Smart Cloud (FCI) Co., Ltd.
7th Floor, FPT Tower, No. 10 Pham Van Bach Street, Cau Giay District, Hanoi.