Participants discussed the problem statement submitted by Peter de Vocht:

I’m passionate about information overload management. Making information interactive would be my dream. I’ve already worked out methods to perform word sense disambiguation (WSD) using neural networks. This solves problems in information retrieval (IR) by making information more relevant in targeted searching. I am now investigating how to better present / store information to help people “wade” through it.

Making information interactive

Peter provided a demonstration of the semantic search technology he is developing, and outlined how the technology makes use of neural networks to extract semantic similarities and equivalences from a larger body of textual information, which in turn allows the construction of search functionality that focuses on semantic similarity.

Two key underlying assumptions are:

1. That large bodies of textual information, as for example much of the textual information produced by humans on the Web, contain inherent semantics that are of significant value to humans and human organisations.
2. That a finely tuned semantic search function will assist humans reducing the effort needed to unlock some of this value.

Jorn pointed out that the spoken and written human language is a linear serialisation format for snippets of human mental models, and that any translation from mental models into spoken and written human language involves some level of information loss, which in turn limits the potential value that is available to information mining tools. Semantic search as outlined by Peter assumes a very literal/autistic interpretation of written language – that people communicate to exchange knowledge, and that other motivations such as deception and the playing of social games play a negligible role. These assumptions may be reasonable in some contexts and less so in others.

Semantics and Natural Language Understanding

Language is difficult, as are semantics. Semantics seem to be arbitrary. Jorn too discussed the difficulties of “personalized” semantics, i.e. semantics being unique to individuals.

The difficulty in language stems from its ambiguity, and its abstract representation of information. The information represented in text, too, is a step removed from reality.

Semantics (or the relationships of language) can help with searching and information retrieval. As we use the relationships of language (e.g. synonyms) we can further expand indexes and provide something I call “language invariance”. Although there is a cost in doing this (the cost being extra storage and ultimately adding to information overload), the language invariance means that the same information described with different words can often be found.

Vice versa, semantics can resolve ambiguity by distinguishing between ambiguous terms and allowing more precise targeting of information. For instance, if we could search for “bank (the financial institution kind)” instead of just “bank” we could cut out other meanings of bank (e.g. “river banks”).

Natural Language Understanding (NLU)

This is all very interesting, but our goal really is to make information interactive. Semantics help information retrieval (a little). As information grows we still are stuck with the same problems. Too much information and potentially more of it due to semantic expansions.

What if we could understand queries and answer questions instead of doing a “search”?

For this to happen we need to look at the information more deeply and differently.

The first form of NLU I discussed was the verb approach where the verb in a sentence is the “function” and the other items of the sentence are the parameters of that function. This comes in many forms (e.g. see VerbNet).

The central idea is that there are only a limited number of these functions. Instead of the 25,000 verbs that span English (roughly) we could reduce this set to a proto-typical set of around 180 verbs (see VerbNet’s predicates).

However, then we’re still missing the semantics. We can have lexicons with the names of entities and their markup (e.g. “London isa city”, “Jorn isa person”, etc.). That helps, and that enables us to further mark items as being of specific semantics.

One method to process sentences in terms of verbs and parameters is called Semantic Role Labelling (SRL). This identifies “objects”, “subjects”, and “verbs” in sentences. The reason these roles (e.g. “object”) are so over general is that most language can’t be pinned down to more exact parts.

One can generate trillions of sentences with a language like English. A recent study showed too that creating 3-grams (3 word combinations of every word in a large set of text, always sequentially) – those 3-grams when compared to other random texts only cover 70% of other texts. What this means is that language is so generative and massive, that even combinations of large sets of three words “aren’t seen again” in other large sets of text.

So what next?

NLU and ANN (Artificial Neural Networks)

There is actually a solution that is very elegant for the semantics of words. This is called word2Vec.

Basically a word2vec is a unique vector of 400 to 4000 float items (the number is fixed once its chosen, the length of the vector is just how many different “semantics” you want to represent). What these floats in the vectors mean is actually unknown. All they end up representing is a concept in a high dimensional space.

The vectors two are different each time they are re-generated as ANNs are basically randomly initialised at the start.

Google has its own set of “news vectors” that are downloadable for a large set of words, however, these semantics too are domain specific.

What is interesting is that the following holds for the vectors of the following words:

                                    Queen – woman + man =~ King

The way we calculate this is through the cos angle of the dot-product of the vectors, giving a similarity between vectors from 0 degrees (identical) to 90 degrees (least related / greatest distance).

Towards interactivity

So what next? Can we tackle this problem?

Perhaps there is a way. We talked about intelligent limited agents. The limits being akin to the limits of a human being just to keep it natural. Perhaps such an agent is a “shadow” or a “twin” of a person (if we ever can get that sophisticated).

There are a few things I’m going to investigate next (when I get the time)

• intelligent indexes. What if the indexes themselves gather more information and became primitive agents themselves?
• identifying irrelevant data. Using neural networks to remove data that never needs to be indexed because it represents very little or no information.

Keynote talk

Leading up to the lunch break Jorn presented a keynote (slides) on the relationship between neurodiversity and creativity, the impact of widespread discrimination against people with autistic traits in the workplace, and on the need for radical autistic activism.

Group discussion

Participants then compared two fundamentally different approaches to knowledge sharing and knowledge extraction. The discussion was interwoven with further discussion of neurodiversity and the role of different cognitive lenses in shaping the motivations behind humans communication, as well as the preferred forms of expression and interaction.

Perspective A: The perspective embodied by Peter’s approach to semantic search, which relies on the production of human language artefacts by a specific group or community, and which then proceeds by:

1. Encoding basic assumptions about human language (the categories of verbs, nouns, glue, etc.) and language use within the community (all artefacts are assumed to be in one language, i.e. English).
2. Training a neural network on the corpus of information created by the group or community to extract semantic equivalences.
3. Offering an elegant semantic search user interface that accepts human language as input and makes use of semantic similarity to compile a set of “good” matches, with the main goal of eliminating irrelevant results (matches of words or word sequences, but no semantic match) and of including all relevant results (by including semantic matches between different words or word sequences).

Perspective B: The perspective embodied by S23M’s approach to semantic modelling, which relies on the production of formal semantic domains and models (typically expressed in a visual notation) by specific individuals, and which then proceeds by:

1. Joint validation the semantic identities used in the formal models via instantiation in the context of a small group or community.
2. Formal recording of the level of shared understanding in relation to a specific semantic domain. This is achieved via declaration of all semantic equivalences that exist between semantic identities (concepts) across the models used by different people, and can be reinforced by collaborative symbol/icon development relating to the core of shared understanding.
3. Allowing each individual to decorate semantic identities with their preferred textual labels, and automatically translating all the textual labels according to individual preferences when sharing semantic domains and models.

Perspective A is useful when dealing with large established bodies of human language artefacts. It results in semantic models that are limited by the same constraints that apply when humans interact exclusively via written language, leaving it to further human interpretation to infer motivation and to detect potential attempts of deception. A core strength of the approach lies in its ability to ingest huge volumes of existing human language artefacts, and in assisting humans in navigating this content (the “socialisation” activity within the SECI knowledge creation process). The main limitation of the approach lies in the textual substrate from which semantics are extracted, which lacks meta information about the concrete context in which the text was produced (beyond the basics of author, time, and location that may be available).

Perspective B is useful when dealing with humans who posses significant amounts of tacit knowledge in relation to a specific domain or field of expertise. It results in formal semantic models that have been validated interactively via instantiation by two or more domain experts with complementary knowledge and experiences. The process of validation via instantiation is an effective tool for uncovering misunderstandings and attempts of deception between humans. A core strength of the approach lies in its recognition of interactivity between humans as a cornerstone of creating shared understanding and new knowledge, and in supporting all activities within the SECI knowledge creation process (“externalisation”, “combination”,  “internalisation”, “socialisation”). The main limitations of the approach are:

• its heavy reliance on human tacit knowledge to prime the knowledge creation pump
• and its inability to directly extract potential value that may be encoded in large established bodies of human language artefacts.

At the risk of slight over-simplification, perspective A is concerned with making sense of the rapidly growing volume of human language artefacts produced by larger groups of (potentially anonymous) human agents, whereas perspective B is concerned with accelerating the learning and trust building process within smaller groups of collaborating human agents.

Both perspectives are complementary. Perspective A pushes/extends the limits of the familiar paradigm of communication using linear human languages, which has been in use for at least 100,000 years. Perspective B represents a relatively new paradigm for communication based on visual formal languages, which has become accessible on computing devices only within the last 20 years, following the advent of ubiquitous graphical user interfaces – motivated by the success of numerous visual domain specific notations of varying levels of formality in architecture, chemistry, engineering, and mathematics.

It would be great to see a continuation of this dialogue at one of the upcoming CIIC unconferences in 2017!