Validation report detect aberration

This method tested for morphological differences between anomalous and normal.

Probes were imaged and images were segmented into regular 128x128 pixel segments. Artificial Neural Networks (ANN) were trained to distinguish segments from condition anomalous from segments from condition normal.

To ensure what was learned was not to distinguish specific instances of CELL LINE and their distribution between the classes, we trained one ANN per CELL LINE. We excluded all image segments from that CELL LINE from the training, and then used each ANN to compute scores for all the image segments excluded in their respective training.

Experimental Conditions

The data was separated into conditions anomalous and normal as follows:

MEASUREMENT
POSITION
DATE
VESSEL TYPE
VESSEL
MEASURED ON
CELL LINE
CLASS_NAME
VALID
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

Identifying Dimensions

The classes are completely defined by the values in the metadata dimensions CLASS_NAME or CELL LINE. Each of these dimensions may explain any differences VAIDR may have found in the image data unless it was selected as the grouping criterion (see above).

DIMENSION
ANOMALOUS
NORMAL
1
2

Progress

DONE
MISSING
1
2
3
4

Results

Download scores and metadata as JSON or CSV.

Plots

Mean Image Segment Scores

For each cut-out, a score was computed, which reflects the probability to belong to one of the two classes. A score below 0.5 means a classification into the first class, above 0.5 means classification into the second class. For each well, the score histogram is represented as a violin plot. Additionally, scores were aggregated for all wells by selecting the median score of all cut-outs. Median well scores are shown as circular markers. The wells can be grouped by selectable conditions and the circular markers can be turned into pie-charts, reflecting selectable conditions.

Classes anomalous, normal | Group by
| | Pie Charts
1251579714 - B1 1251579714 - B3 1789167944 - A2 507290396 - A1 507290396 - A3 1283639645 - A2 490480743 - A1 490480743 - A3 84458067 - A2 1283639645 - B1 1283639645 - B3 490480743 - B2 84458067 - B1 84458067 - B3 1251579714 - A2 1789167944 - B1 1789167944 - B3 507290396 - B2 1040103636 - A1 1040103636 - A3 1040103636 - B2 1127785965 - A1 1127785965 - A3 1127785965 - B2 2084470135 - A1 2084470135 - A3 2084470135 - B2 1074377409 - A1 1074377409 - A3 1074377409 - B2 1570617141 - A1 1570617141 - A3 1570617141 - B2 678835707 - A1 678835707 - A3 678835707 - B2 measurement - well 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 mean classification score iPSC 002 Left out group: iPSC 002 iPSC 003 Left out group: iPSC 003 iPSC 005 Left out group: iPSC 005 iPSC 001 Left out group: iPSC 001 iPSC 005 Left out group: iPSC 005 iPSC 004 Left out group: iPSC 004 Measurement 1251579714 B1 Measurement 1251579714 B2 Measurement 1251579714 B3 Measurement 1789167944 A1 Measurement 1789167944 A2 Measurement 1789167944 A3 Measurement 507290396 A1 Measurement 507290396 A2 Measurement 507290396 A3 Measurement 1283639645 A1 Measurement 1283639645 A2 Measurement 1283639645 A3 Measurement 490480743 A1 Measurement 490480743 A2 Measurement 490480743 A3 Measurement 84458067 A1 Measurement 84458067 A2 Measurement 84458067 A3 Measurement 1283639645 B1 Measurement 1283639645 B2 Measurement 1283639645 B3 Measurement 490480743 B1 Measurement 490480743 B2 Measurement 490480743 B3 Measurement 84458067 B1 Measurement 84458067 B2 Measurement 84458067 B3 Measurement 1251579714 A1 Measurement 1251579714 A2 Measurement 1251579714 A3 Measurement 1789167944 B1 Measurement 1789167944 B2 Measurement 1789167944 B3 Measurement 507290396 B1 Measurement 507290396 B2 Measurement 507290396 B3 Measurement 1040103636 A1 Measurement 1040103636 A2 Measurement 1040103636 A3 Measurement 1040103636 B1 Measurement 1040103636 B2 Measurement 1040103636 B3 Measurement 1127785965 A1 Measurement 1127785965 A2 Measurement 1127785965 A3 Measurement 1127785965 B1 Measurement 1127785965 B2 Measurement 1127785965 B3 Measurement 2084470135 A1 Measurement 2084470135 A2 Measurement 2084470135 A3 Measurement 2084470135 B1 Measurement 2084470135 B2 Measurement 2084470135 B3 Measurement 1074377409 A1 Measurement 1074377409 A2 Measurement 1074377409 A3 Measurement 1074377409 B1 Measurement 1074377409 B2 Measurement 1074377409 B3 Measurement 1570617141 A1 Measurement 1570617141 A2 Measurement 1570617141 A3 Measurement 1570617141 B1 Measurement 1570617141 B2 Measurement 1570617141 B3 Measurement 678835707 A1 Measurement 678835707 A2 Measurement 678835707 A3 Measurement 678835707 B1 Measurement 678835707 B2 Measurement 678835707 B3

Binary Classification Scores

To test whether it was possible to distinguish wells assigned to anomalous from wells assigned to class normal, aggregate scores for each well were computed by selecting the median scores for each of the classes across all cut-outs belonging to that well. A classification score was computed for each well by dividing the aggregate score for anomalous by the sum of the aggregate scores for anomalous and normal. Classification scores were used for a Box Plot and a ROC Curve Plot.

Classes anomalous, normal | Group by
| | Pie Charts
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 anomalous Binary Scores *** anomalous normal class anomalous normal 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 score Measurement 84458067 B3 Measurement 84458067 A2 Measurement 490480743 B2 Measurement 84458067 B2 Measurement 84458067 A3 Measurement 84458067 B1 Measurement 84458067 A1 Measurement 490480743 A1 Measurement 490480743 A3 Measurement 490480743 B3 Measurement 490480743 A2 Measurement 490480743 B1 Measurement 507290396 A1 Measurement 1251579714 B1 Measurement 1251579714 B2 Measurement 1251579714 B3 Measurement 1283639645 A2 Measurement 1789167944 A1 Measurement 1283639645 A1 Measurement 507290396 A2 Measurement 1283639645 B2 Measurement 1283639645 A3 Measurement 507290396 A3 Measurement 1789167944 B1 Measurement 1789167944 A3 Measurement 1283639645 B1 Measurement 1789167944 A2 Measurement 2084470135 B3 Measurement 1789167944 B2 Measurement 1127785965 A3 Measurement 2084470135 B2 Measurement 1127785965 B2 Measurement 1127785965 A1 Measurement 1789167944 B3 Measurement 1127785965 B3 Measurement 2084470135 A3 Measurement 1127785965 B1 Measurement 1283639645 B3 Measurement 1127785965 A2 Measurement 2084470135 B1 Measurement 1040103636 A2 Measurement 2084470135 A1 Measurement 1040103636 A1 Measurement 2084470135 A2 Measurement 507290396 B3 Measurement 1040103636 B2 Measurement 1251579714 A2 Measurement 1570617141 A2 Measurement 1040103636 B3 Measurement 1570617141 A3 Measurement 1040103636 B1 Measurement 507290396 B1 Measurement 1570617141 B2 Measurement 507290396 B2 Measurement 1570617141 B3 Measurement 1251579714 A3 Measurement 1040103636 A3 Measurement 678835707 B2 Measurement 678835707 A3 Measurement 678835707 B3 Measurement 1251579714 A1 Measurement 1570617141 B1 Measurement 1570617141 A1 Measurement 678835707 A2 Measurement 1074377409 A1 Measurement 1074377409 B1 Measurement 1074377409 B3 Measurement 1074377409 A3 Measurement 1074377409 A2 Measurement 1074377409 B2 Measurement 678835707 A1 Measurement 678835707 B1 Binary Score ROC Curve 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 score quantile normal 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 score quantile normal 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 score quantile anomalous 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 score quantile anomalous Measurement 84458067 B3 Measurement 84458067 A2 Measurement 490480743 B2 Measurement 84458067 B2 Measurement 84458067 A3 Measurement 84458067 B1 Measurement 84458067 A1 Measurement 490480743 A1 Measurement 490480743 A3 Measurement 490480743 B3 Measurement 490480743 A2 Measurement 490480743 B1 Measurement 507290396 A1 Measurement 1251579714 B1 Measurement 1251579714 B2 Measurement 1251579714 B3 Measurement 1283639645 A2 Measurement 1789167944 A1 Measurement 1283639645 A1 Measurement 507290396 A2 Measurement 1283639645 B2 Measurement 1283639645 A3 Measurement 507290396 A3 Measurement 1789167944 B1 Measurement 1789167944 A3 Measurement 1283639645 B1 Measurement 1789167944 A2 Measurement 2084470135 B3 Measurement 1789167944 B2 Measurement 1127785965 A3 Measurement 2084470135 B2 Measurement 1127785965 B2 Measurement 1127785965 A1 Measurement 1789167944 B3 Measurement 1127785965 B3 Measurement 2084470135 A3 Measurement 1127785965 B1 Measurement 1283639645 B3 Measurement 1127785965 A2 Measurement 2084470135 B1 Measurement 1040103636 A2 Measurement 2084470135 A1 Measurement 1040103636 A1 Measurement 2084470135 A2 Measurement 507290396 B3 Measurement 1040103636 B2 Measurement 1251579714 A2 Measurement 1570617141 A2 Measurement 1040103636 B3 Measurement 1570617141 A3 Measurement 1040103636 B1 Measurement 507290396 B1 Measurement 1570617141 B2 Measurement 507290396 B2 Measurement 1570617141 B3 Measurement 1251579714 A3 Measurement 1040103636 A3 Measurement 678835707 B2 Measurement 678835707 A3 Measurement 678835707 B3 Measurement 1251579714 A1 Measurement 1570617141 B1 Measurement 1570617141 A1 Measurement 678835707 A2 Measurement 1074377409 A1 Measurement 1074377409 B1 Measurement 1074377409 B3 Measurement 1074377409 A3 Measurement 1074377409 A2 Measurement 1074377409 B2 Measurement 678835707 A1 Measurement 678835707 B1

The mean score for wells from anomalous was significantly smaller than that for normal (p = 0.00000 < 0.001)

Statistics on Population Score Means

As a simple means to determine whether our ANNs were able to learn to distinguish the conditions reliably and robustly, we computed the mean image segment scores per population (well/flask) and compared them to the threshold of 0.5.

Populations with a mean score ≥ 0.5 and < 0.5 were counted classified as anomalous and normal, respectively. From this classification we were able to compute prediction accuracy.

Prediction accuracy was 0.94. The base rate that would have been achieved by always selecting the majority class was 0.63.

Statistically, this result is highly significant (p ≪ 0.001).

The best possible p-value given this number of conditions (72) and base rate would have been p = 0.00000.

Interpretation

We can conclude that the ANNs we trained learned to distinguish between image segments from populations from anomalous and normal. Barring alternative effects which may have systematically affected cell morphology, this result supports the hypothesis that cells from the two conditions do have morphological differences.