ISTQB AI Testing | Quality House Ltd.

Duration

ISTQB Specialist Level AI Testing Course is a three-day course, usually held in
the time period Friday - Sunday. Each training day starts at 9.00 and finishes at 18.00.

Location

Quality House is flexible and focuses on the needs of our clients. Public courses are housed in a hall provided by Quality House for the official dates in our Courses Calendar. Our company also organizes private courses for a single client, where both the dates and location are further negotiated. The client can suggest and provide a convenient location suited for their needs.

Accreditation

The ISTQB Specialist Level AI Testing Course meets International Software Testing Qualifications Board (ISTQB) requirements and has been accredited by South East European Testing Board (SEETB).

Course Content

Introduction to AI

Definition of IA and AI Effect
Narrow, General and Super AI
AI-Based and Conventional Systems
AI Technologies
AI Development Frameworks
Hardware for AI-Based Systems
AI as a Service (AIaaS)
Pre-Trained Models
Standards, Regulations and AI

Quality Characteristic for AI Based Systems

Flexibility and Adaptability
Autonomy
Evolution
Bias
Ethics
Side Effects and Reward Hacking
Transparency, Interpretability and Explainability
Safety and AI

Machine Learning (ML) – Overview

Forms of ML
ML Workflow
Selecting a Form of ML
Factors Involved in ML Algorithm Selection
Overfitting and Underfitting

ML Data

Data Preparation as Part of the ML Workflow
Training, Validation and Test Datasets in the ML Workflow
Dataset Quality Issues
Data Quality and its Effect on the ML Model
Data Labelling for Supervised Learning

ML Functional performance Metrics

Confusion Matrix
Add ML Functional performance Metrics for Classification, Regression and Clustering
Limitations of ML Functional Performance Metrics
Selecting ML Functional Performance Metrics
Benchmark Suites for ML Performance

ML Neural Networks and Testing

Neural Networks
Coverage Measures for Neural Networks

Testing AI-Based Systems – Overview

Specification of AI-Based Systems
Test Levels of AI-Based systems
Test Data for Testing AI-Based Systems
Testing for Automation Bias in AI-Based Systems
Documenting an AI-Based Component
Testing for Concept Drift
Selecting a Test Approach for an ML System

Testing AI-Specific Quality Characteristics

Challenges Testing Self-Learning Systems
Testing Autonomous Self-Learning Systems
Testing for Algorithmic, Sample and Inappropriate Bias
Challenges Testing Probabilistic and Non-Deterministic AI-Based Systems
Challenges Testing Complex AI-Based Systems
Testing Transparency Interpretability and Explainability of AI-Based Systems
Test Oracles for AI-Based Systems
Test objectives and Acceptance Criteria

Methods and Techniques for the Testing of AI-Based Systems

Adversarial Attacks and Data Poisoning
Pairwise Testing
A/B Testing
Back-to-Back Testing
Metamorphic Testing (MT)
Experience-Based Testing of AI-Based Systems
Selecting Test Techniques for AI-Based Systems

Testing Environments for AI-Based Systems

Test Environments for AI-Based Systems
Virtual Test Environments for Testing AI-Based Systems

Using AI for Testing

AI Technologies for Testing
Using AI to Analyze Defect Reports
Using AI to Test Case Generation
Using AI for the Optimization of Regression Test Suites
Using AI for Defect Prediction
Using AI for Testing User Interfaces

Course Benefits

Understand the current state and expected trends of AI
Experience the implementation and testing of a ML model and recognize where testers can best influence its quality
Understand the challenges associated with testing AI-Based systems, such as their self-learning capabilities, bias, ethics, complexity, non-determinism, transparency and explainability
Contribute to the test strategy for an AI-Based system
Design and execute test cases for AI-based systems
Recognize the special requirements for the test infrastructure to support the testing of AI-based systems
Understand how AI can be used to support software testing

Disclaimers

The Specialist Level - AI Testing Certification Exam has the form of a multiple - choice test and consists of 40 questions in English.
Each correct answer brings one point. Passing score - 65% or 31 points (maximum possible points - 47). Duration of the test - 75 minutes for non-native speakers.
To take the exam, you should hold the ISTQB Foundation Level certificate.
The exam administration and acquisition of the ISTQB certificates are managed by South East European Testing Board (SEETB), a trusted Quality House partner, resulting in a supplement fee.
The course requires a minimum number of participants. If the required group size is not reached, we will offer you the opportunity to reschedule for alternative dates or cancel if preferred.