What's new in June 2020

Better Language Models, Automated training (with Kubeflow), Keyboard Hints, Interface Insights, Message Tags and Research & survey

Better Language Models

Earlier this year we’ve experimented with advanced transformer models like GPT-2 for text generation.

Experimental

We now have deployed a GPT-2 model fine tuned to customer data in production with a number of agents. Being experimental, we have seen some grammar issues (often typo’s that occur frequently in the training data as well) and we have fixed a small glitch with disappearing suggestions.

Promising 🤱🏼

That being said, first results are promising. The model produces an additional 5-8% Assisted text. And on average 15% of the suggestions are used giving a sense of the quality of the recommendations. We will be releasing this gradually into production to become available to all users in the near future.

Automated training 🗳

At Deepdesk we have multiple machine learning pipelines per customer. These pipelines include multiple steps like preprocessing, clustering and training.

Kubeflow

In order to manage all this we now use Kubeflow in combination with Kubernetes. This allows us to schedule automated machine learning pipelines, as illustrated below.

Auto A/B

Pipelines now run periodically with the following
autonomous steps: select correct dataset, preprocess data, cluster & validate, send training report, monitor redactions, deploy models in A/B, evaluate and run best models 🙌🏻

Automation

The automation of these pipeline enables us to train more frequently, resulting in more up to date suggestions. As an added bonus this automation gives us more time to develop new features... 🧁

Keyboard hints ⌨️

We introduce keyboard hints to promote the tab or arrow keys for selecting suggestions. This allows agents to keep their hands on the keyboard, and select suggestions faster. With faster selection we anticipate increased overall usage.

Interface insights

In order to get a better understanding of how agents select suggestions we implemented some additional tracking. For every used suggestion we can now see if this was done with tab, arrow keys or mouse, allowing for optimisation in how we offer suggestions.🧑🏽‍💻

Message tags

We redesigned our data model to store message properties more flexible. With new requirements, new platforms or additional message details we needed a more flexible model to store the details. Tags are used to determine which model to use, and can be used for selecting which conversations to train on.🍒

Research & survey

We have conducted an extensive user survey in combination with core product research. We will use the valuable feedback on user experience and actionable insights to determine our future roadmap and create additional improvements for our core product. Thanks to all who participated. 🙏🏼