Creation and Evaluation of Tacit Knowledge Based on Semantic Primes

点击翻译按钮获取中文摘要

During the first year I have concentrated on developing the first set of tacit knowledge for further experiments. I considered the limited number of semantic primes to answer the question "which types of cognitive functionality an agent should posses before exploring the environment and understanding or learning about the world?". Inspired by the exponents grouped into related semantic primitives categories and their translation to Japanese, I created a list of perception-related 23 types of prompts. After series of many preliminary experiments and annotator tests, I simplified, aggregated and specified some prompts to make the annotation shorter and easier.In order to prepare the final dataset I utilized a Japanese sentences from the previous project. Meant for detecting danger level changes in slightly different contexts, it comprises of short sentence pairs as "child eats a soap" and "child eats a soap-shaped candy". I have created a program for generating prompts to make queries about agents, patients and acts in context of semantic primes and hired 66 annotators to choose answers about a sentence.The final golden set for future experiments consisted of 62,687 annotated sentence-prompt-choice triples and has been described in detail in an international conference paper which is currently under review. Experimental results show that although the agreement between annotators was high, classic language models as BERT and RoBERTa performed poorly in a task of recognizing cognitive perception-related questions based on semantic primes.