Skip to content

Characteristics - details

This section contains the following information for each characteristic:

  • Definition
  • Why is this characteristic important?

Robustness

Definition

Level of degradation suffered by the ML System under consideration when exposed to dynamic or adverse events.

Why is Robustness important?

An ML System should not require disproportionate maintenance in day to day operations and also in face of changes in the environment (such as increased traffic).

Modifiability

Definition

The degree of effectiveness and efficiency with which an ML System can be modified to improve it, correct it or adapt it to changes in environment, and in requirements.

Why is Modifiability important?

A Modifiable ML System delivers value in the long term, when new use cases arise, or when the environment changes and the ML System needs to be adapted. It is important new changes can be implemented with low cost.

Economy

Definition

Level of performance relative to the amount of resources used under stated conditions.

Why is Economy important?

It is important to consider both the cost to impact ratio for an ML System to make sure the system is worth deploying as well as to consider the cost of training, deployment, and inference.

Utility

Definition

The degree to which an ML System provides functions that meet stated and implied needs when used under specified conditions.

Why is Utility important?

An ML systems needs to be useful, in order to bring value to the end user. Utility is about those aspects which make an ML system useful for the end users.

Comprehensibility

Definition

The degree to which users and contributors understand the relevant aspects of an ML system.

Why is Comprehensibility important?

Comprehensibility is about building ML systems that are easy to be understood, used and debugged. Comprehensibility refers both to contributors and users, since some sub-characteristics like Usability refer to the user while others like Debuggability refer to the contributor. A system that is not comprehensible might have limited impact, due to the lack of transparency regarding its usage.

Productionizability

Definition

The ease with which the actions needed for an ML system to operate successfully in production can be performed.

Why is Productionizability important?

An ML system can have value only if it being used in production for a certain use case. Lack of Productionizability means that it is hard to fully exploit the model in production something that might result in limited value.

Responsibility

Definition

The level of trustworthiness of an ML system.

Why is Responsibility important?

The value of an ML system does not only come from its performance on a business metric, but also from its trusthworthiness, e.g. the level of ownership, security, fairness and transparency. A system that is not trustworthy, might end up not being used as it might pose risks that the business cannot afford.