For each document, there is one single answer which is considered to be the one and only correct answer. We label this answer as the version "gold".
Versions of the answer
However, different versions by different people may not be identical, even if they are instructed to provide correct data. This can have different root causes
- The definition of correct can be ambiguous
- The definition of correct can evolve over time
- People make mistakes
Versions allow us to keep track of different people's view on the correct answer.
Furthermore, versions allow us to track the machine's view on the best possible prediction over time.
Evaluation of a version
So far, we have talked about versions of the answer. Each version of the answer can then be compared against the answer marked as "gold". This is what we call the evaluation of a version.
Version tags
Each version has a tag to identify the version. This tag is set automatically for the first prediction ("initial_prediction") and the first evaluation by a person ("first_pass"). Additional versions can be created, typically through the /wiki/spaces/DA/pages/347013238 pane. The are assigned a label when the documents are scheduled for an additional version (by the ML pipeline or by a human operator).
Visualisation of versions
As such, versions are mostly visible in two locations
- In the data entry companion - to visualise a specific answer (by the machine or by a person)
- In the statistics pane - to visualise the evaluation of a specific version against "gold"
Illustration: You can browse between the different versions in the Data Entry Companion using the Version menu.
Illustration: You can select the evaluation of a given version in the stats pane.
