Classification with GPT-3

In the following, we use GPT-3 to label abstracts as pertaining to one or more subfields. First, we will use the vanilla model to test to what extent its ‘world knowledge’ allows it to perform the task at hand. The results are surprisingly good, yet the accuracy of the labeling is not yet good enough to be used as the basis of a scientometric study.

Hence, we next check whether and to what extent this result can be improved by fine-tuning the model on our specific problem and data. As it turns out, we achieve excellent results with this approach with only moderate effort.