iSQI CT-AI Certified Tester AI Testing Exam Practice Test

Page: 1 / 14
Total 80 questions

Want more questions? Get Premium Access.
()

Question 1

"AllerEgo" is a product that uses sell-learning to predict the behavior of a pilot under combat situation for a variety of terrains and enemy aircraft formations. Post training the model was exposed to the real-

world data and the model was found to be behaving poorly. A lot of data quality tests had been performed on the data to bring it into a shape fit for training and testing.

Which ONE of the following options is least likely to describes the possible reason for the fall in the performance, especially when considering the self-learning nature of the Al system?

SELECT ONE OPTION

AThe difficulty of defining criteria for improvement before the model can be accepted.

BThe fast pace of change did not allow sufficient time for testing.

CThe unknown nature and insufficient specification of the operating environment might have caused the poor performance.

DThere was an algorithmic bias in the Al system.

Answer : A

A . The difficulty of defining criteria for improvement before the model can be accepted.

Defining criteria for improvement is a challenge in the acceptance of AI models, but it is not directly related to the performance drop in real-world scenarios. It relates more to the evaluation and deployment phase rather than affecting the model's real-time performance post-deployment.

B . The fast pace of change did not allow sufficient time for testing.

This can significantly affect the model's performance. If the system is self-learning, it needs to adapt quickly, and insufficient testing time can lead to incomplete learning and poor performance.

C . The unknown nature and insufficient specification of the operating environment might have caused the poor performance.

This is highly likely to affect performance. Self-learning AI systems require detailed specifications of the operating environment to adapt and learn effectively. If the environment is insufficiently specified, the model may fail to perform accurately in real-world scenarios.

D . There was an algorithmic bias in the AI system.

Algorithmic bias can significantly impact the performance of AI systems. If the model has biases, it will not perform well across different scenarios and data distributions.

Given the context of the self-learning nature and the need for real-time adaptability, option A is least likely to describe the fall in performance because it deals with acceptance criteria rather than real-time performance issues.

Question 2

Which ONE of the following options is an example that BEST describes a system with Al-based autonomous functions?

SELECT ONE OPTION

AA system that utilizes human beings for all important decisions.

BA fully automated manufacturing plant that uses no software.

CA system that utilizes a tool like Selenium.

DA system that is fully able to respond to its environment.

Answer : D

AI-Based Autonomous Functions: An AI-based autonomous system is one that can respond to its environment without human intervention. The other options either involve human decisions or do not use AI at all.

Reference: ISTQB_CT-AI_Syllabus_v1.0, Sections on Autonomy and Testing Autonomous AI-Based Systems.

Question 3

A transportation company operates three types of delivery vehicles in its fleet. The vehicles operate at different speeds (slow, medium, and fast). The transportation company is attempting to optimize scheduling and has created an AI-based program to plan routes for its vehicles using records from the medium-speed vehicle traveling to selected destinations. The test team uses this data in metamorphic testing to test the accuracy of the estimated travel times created by the AI route planner with the actual routes and times.

Which of the following describes the next phase of metamorphic testing?

AThe team tests the time required for the fast and slow vehicles to travel the same route as the medium vehicle. Then, by calculating the speed difference, they then predict how much faster or slower the vehicles will travel. That information is then used to verify that the arrival time of the vehicles meets the expected result.

BThe team decomposes each route into the relevant components that affect the travel time such as traffic density and vehicle power. The team then uses statistical analysis to characterize the influence of each component to calculate the fast and slow vehicle route times.

CThe team uses an AI system to select the most dissimilar routes. With this information, any of the AI routes can be metaphorically transformed into a fast or slow route.

DThe team uses the same AI route planner to create routes that are longer and shorter but follow the same track. Finally, by driving the fast vehicles on the long routes and slow vehicles on the short routes and vice versa, the AI system will have enough information to infer travel times for all vehicles on all routes.

Answer : A

Metamorphic Testing (MT) is a testing technique that verifies AI-based systems by generating follow-up test cases based on existing test cases. These follow-up test cases adhere to a Metamorphic Relation (MR), ensuring that if the system is functioning correctly, changes in input should result in predictable changes in output.

Why Option A is Correct?

Metamorphic testing works by transforming source test cases into follow-up test cases

Here, the source test case involves testing the medium-speed vehicle's travel time.

The follow-up test cases are derived by extrapolating travel times for fast and slow vehicles using predictable relationships based on speed differences.

MR states that modifying input should result in a predictable change in output

Since the speed of the vehicle is a known factor, it is possible to predict the new arrival times and verify whether they follow expected trends.

This is a direct application of metamorphic testing principles

In route optimization systems, metamorphic testing often applies transformations to speed, distance, or conditions to verify expected outcomes.

Why Other Options are Incorrect?

(B) Decomposing each route into traffic density and vehicle power

While useful for statistical analysis, this approach does not generate follow-up test cases based on a defined metamorphic relation (MR).

This does not follow metamorphic testing principles, which require predictable transformations.

(D) Running fast vehicles on long routes and slow vehicles on short routes

This method does not maintain a controlled MR and introduces too many uncontrolled variables.

Reference from ISTQB Certified Tester AI Testing Study Guide

Metamorphic testing generates follow-up test cases based on a source test case. 'MT is a technique aimed at generating test cases which are based on a source test case that has passed. One or more follow-up test cases are generated by changing (metamorphizing) the source test case based on a metamorphic relation (MR).'

MT has been used for testing route optimization AI systems. 'In the area of AI, MT has been used for testing image recognition, search engines, route optimization and voice recognition, among others.'

Thus, option A is the correct answer, as it aligns with the principles of metamorphic testing by modifying input speeds and verifying expected results.

Question 4

Which of the following is a technique used in machine learning?

ADecision trees

BEquivalence partitioning

CBoundary value analysis

DDecision tables

Answer : A

Decision trees are a widely used machine learning (ML) technique that falls under supervised learning. They are used for both classification and regression tasks and are popular due to their interpretability and effectiveness.

How Decision Trees Work:

The model splits the dataset into branches based on feature conditions.

It continues to divide the data until each subset belongs to a single category (classification) or predicts a continuous value (regression).

The final result is a tree structure where decisions are made at nodes, and predictions are given at leaf nodes.

Common Applications of Decision Trees:

Fraud detection

Medical diagnosis

Customer segmentation

Recommendation systems

Why Other Options Are Incorrect:

B (Equivalence Partitioning): This is a software testing technique, not a machine learning method. It is used to divide input data into partitions to reduce test cases while maintaining coverage.

C (Boundary Value Analysis): Another software testing technique, used to check edge cases around input boundaries.

D (Decision Tables): A structured testing technique used to validate business rules and logic, not a machine learning method.

Supporting Reference from ISTQB Certified Tester AI Testing Study Guide:

ISTQB CT-AI Syllabus (Section 3.1: Forms of Machine Learning - Decision Trees)

'Decision trees are used in classification and regression models and are fundamental ML algorithms'.

Conclusion:

Since decision trees are a core technique in machine learning, while the other options are software testing techniques, the correct answer is A.

Question 5

Data used for an object detection ML system was found to have been labelled incorrectly in many cases.

Which ONE of the following options is most likely the reason for this problem?

SELECT ONE OPTION

ASecurity issues

BAccuracy issues

CPrivacy issues

DBias issues

Answer : B

The question refers to a problem where data used for an object detection ML system was labelled incorrectly. This issue is most closely related to 'accuracy issues.' Here's a detailed explanation:

Accuracy Issues: The primary goal of labeling data in machine learning is to ensure that the model can accurately learn and make predictions based on the given labels. Incorrectly labeled data directly impacts the model's accuracy, leading to poor performance because the model learns incorrect patterns.

Why Not Other Options:

Security Issues: This pertains to data breaches or unauthorized access, which is not relevant to the problem of incorrect data labeling.

Privacy Issues: This concerns the protection of personal data and is not related to the accuracy of data labeling.

Bias Issues: While bias in data can affect model performance, it specifically refers to systematic errors or prejudices in the data rather than outright incorrect labeling.

Question 6

An engine manufacturing facility wants to apply machine learning to detect faulty bolts. Which of the following would result in bias in the model?

ASelecting training data by purposely excluding specific faulty conditions

BSelecting training data by purposely including all known faulty conditions

CSelecting testing data from a different dataset than the training dataset

DSelecting testing data from a boat manufacturer's bolt longevity data

Answer : A

Bias in AI models often originates from incomplete or non-representative training data. In this case, if the training dataset purposely excludes specific faulty conditions, the machine learning model will fail to learn and detect these conditions in real-world scenarios.

This results in:

Sample bias, where the training data is not fully representative of all possible faulty conditions.

Algorithmic bias, where the model prioritizes certain defect types while ignoring others.

Why are the other options incorrect?

B . Selecting training data by purposely including all known faulty conditions This would help reduce bias by improving model generalization.

C . Selecting testing data from a different dataset than the training dataset This is a good practice to evaluate model generalization but does not inherently introduce bias.

D . Selecting testing data from a boat manufacturer's bolt longevity data While using unrelated data can create poor model accuracy, it does not directly introduce bias unless systematic patterns in the incorrect dataset lead to unfair decision-making.

Reference from ISTQB Certified Tester AI Testing Study Guide:

Section 8.3 - Testing for Algorithmic, Sample, and Inappropriate Bias states that sample bias can occur if the training dataset is not fully representative of the expected data space, leading to biased predictions.

Question 7

A bank wants to use an algorithm to determine which applicants should be given a loan. The bank hires a data scientist to construct a logistic regression model to predict whether the applicant will repay the loan or not. The bank has enough data on past customers to randomly split the data into a training data set and a test/validation data set. A logistic regression model is constructed on the training data set using the following independent variables:

Gender

Marital status

Number of dependents

Education

Income

Loan amount

Loan term

Credit score

The model reveals that those with higher credit scores and larger total incomes are more likely to repay their loans. The data scientist has suggested that there might be bias present in the model based on previous models created for other banks.

Given this information, what is the best test approach to check for potential bias in the model?

AExperienced-based testing should be used to confirm that the training data set is operationally relevant. This can include applying exploratory data analysis (EDA) to check for bias within the training data set.

BBack-to-back testing should be used to compare the model created using the training data set to another model created using the test data set, if the two models significantly differ, it will indicate there is bias in the original model.

CAcceptance testing should be used to make sure the algorithm is suitable for the customer. The team can re-work the acceptance criteria such that the algorithm is sure to correctly predict the remaining applicants that have been set aside for the validation data set ensuring no bias is present.

DA/B testing should be used to verify that the test data set does not detect any bias that might have been introduced by the original training data. If the two models significantly differ, it will indicate there is bias in the original model.

Answer : A

Bias in an AI system occurs when the training data contains inherent prejudices that cause the model to make unfair predictions. Experience-based testing, particularly Exploratory Data Analysis (EDA), helps uncover these biases by analyzing patterns, distributions, and potential discriminatory factors in the training data.

Analysis of the Answer Options:

Option A: ''Experience-based testing should be used to confirm that the training data set is operationally relevant. This can include applying exploratory data analysis (EDA) to check for bias within the training data set.''

This is the correct answer. EDA involves examining the dataset for bias, inconsistencies, or missing values, ensuring fairness in ML model predictions.

Option B: ''Back-to-back testing should be used to compare the model created using the training data set to another model created using the test data set. If the two models significantly differ, it will indicate there is bias in the original model.''

Back-to-back testing is used for regression testing and to compare versions of an AI system but is not primarily used to detect bias.

Option C: ''Acceptance testing should be used to make sure the algorithm is suitable for the customer. The team can re-work the acceptance criteria such that the algorithm is sure to correctly predict the remaining applicants that have been set aside for the validation data set ensuring no bias is present.''

Acceptance testing focuses on meeting predefined business requirements rather than detecting and mitigating bias.

Option D: ''A/B testing should be used to verify that the test data set does not detect any bias that might have been introduced by the original training data. If the two models significantly differ, it will indicate there is bias in the original model.''

A/B testing is used for evaluating variations of a model rather than for explicitly identifying bias.

ISTQB CT-AI Syllabus Reference:

Bias Testing Methods: 'AI-based systems should be tested for algorithmic bias, sample bias, and inappropriate bias. Experience-based testing and EDA are useful for detecting bias'.

Exploratory Data Analysis (EDA): 'EDA helps uncover potential bias in training data through visualization and statistical analysis'.

Thus, Option A is the best choice for detecting bias in the loan applicant model.