Acceptance testing: Test level that focuses on determining whether to accept the system.*

Adversarial attacks: Where an attacker slightly perturbs (changes) the inputs to a model in order to influence its prediction(s).

AIOps: Industry category for machine learning analytics technology that enhances IT operations analytical capabilities.

Automated reasoning: Advanced area of computer science that is concerned with applying reasoning in the form of logic to computing systems to make inferences automatically.

Automation bias: Cognitive bias that occurs when a human decision-maker favours the recommendations of an automated decision-making system, over other inputs.

Black-box test technique: Test technique based on an analysis of the specification of a component or system.*

Chaos engineering: Discipline of experimenting on a software system in production in order to build confidence in the system’s capability to withstand turbulent and unexpected conditions.

Classification: Machine learning (ML) function that predicts the output class for a given input.

Component testing: Test level that focuses on individual hardware or software components.*

Concept drift: Change in users’ expected predictions from a model that is presented with the same input data.

Data pipeline: Infrastructure supporting an ML algorithm. It includes acquiring data, pre-processing and preparation, training one or more models and exporting the models to production.

Digital twin: The generation or collection of digital data representing a physical object.

Hyper-parameter: Variable set by a human in an ML model, before training the model.

Integration testing: Test level that focuses on interactions between components or systems.

Knowledge representation: Study of how to put knowledge into a form that a computer can reason with.

Neural network: Network of two or more layers of neurons, connected by weighted links with adjustable weights, which takes input data and produces an output. Also called an artificial neural network.

Neuron: Node in a neural network that takes input values and produces an output value, by combining the input values and applying an activation function to the result.

Ontology: Explicit specification of a conceptualisation.

Reasoning: Form of AI that generates conclusions from available information using logical techniques.

Regression: ML function that outputs continuous (typically, floating point) values.

Reinforcement learning: Task of training a model that makes decisions to maximise an objective, using a process of trial and error.

Shift left: Shifting the focus of testing effort towards the early design and testing phase, that is, shifting to the left of the systems or software development cycle.

Shift right: Shifting the focus of testing effort towards testing with users in a production environment.

Sub-symbolic AI: System based on techniques and models, using a numeric representation and implicit information encoding.

Symbolic AI system: System based on techniques and models, using symbols and structures.

System testing: Test level that focuses on verifying that a system as a whole meets specified requirements.*

Test coverage: Degree to which specified coverage items have been determined or have been exercised by a test suite expressed as a percentage.*

Test level: Specific instantiation of a test process.*

Test oracle: A source to determine an expected result to compare with the actual result of the system under test.*

User acceptance testing: Type of acceptance testing performed to determine if intended users accept the system.

Validation: Confirmation, through the provision of objective evidence, that the particular requirements for a specific intended use or application have been fulfilled.*

Verification: Confirmation, through the provision of objective evidence, that specified requirements have been fulfilled.*

White-box test technique: Test technique only based on the internal structure of a component or system.*

XPath: Query language for selecting nodes in XML.

*Copyright © International Software Testing Qualifications Board (hereinafter called ISTQB®).