Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Version allows you to visualise different versions of the data. The most common examples of versions are (1) the initial prediction and (2) the submitted data.

Gold version: For each document, there is one single answer which is considered to be the one and only correct answer. We label this answer as the version "gold".

However, different versions by different people may not be identical, even if they are instructed to provide correct data. This happens because people’s definition of correct can be ambiguous or can evolve over time, or also simply because people make mistakes.

Versions allow us to keep track of (1) different people's view on the correct answer, and (2) the machine's view on the best possible prediction over time.

Evaluation of a version: Each version of the answer can then be compared against the answer marked as "gold". This is what we call the evaluation of a version.

Version tags: Each version has a tag to identify the version. This tag is set automatically for the first prediction ("initial_prediction") and the first evaluation by a person ("first_pass"). Additional versions can be created, typically through the /wiki/spaces/DA/pages/347013238 pane. The are assigned a label when the documents are scheduled for an additional version (by the ML pipeline or by a human operator).

Actions

The actions that you can choose from here are the same as the actions shown previously in the Inbox .

...