Lessons learned from building a commercial open-source bot development platform

As part of the 2nd International Workshop on Bots in Software Engineering (BotSE 2020), I gave the keynote “Lessons learned from building a commercial bot development platform” where I discussed some of the key aspects we have learned while building our opensource and low-code chatbot development platform Xatkit (see also the Xatkit organization in GitHub )

After an introductory mini-tutorial on Xatkit and explaining why the world needs yet another bot solution, I moved to discuss the struggles we are facing when trying to move Xatkit from a research prototype to a commercial-grade tool. Some of the challenges are common to any industrialization effort (when you’re at the same time the research team and the technology provider in charge of maturing the tool) and shared as well with any other core open source project fighting for its long-term sustainability (a topic on which, btw, bots can also play a key role ).

You can check the slides of my talk below but let me also highlight some of our key lessons learned in building a chatbot platform for you, ranging from organizational, technical, and business challenges:

• Organizational challenges
  • You need to be a Polymath to create a bot platform . Plenty of core bot tech: NLU, Sentiment Analysis, Xtext, Slack/GH/… APIs,…., but even more non-bot tech: InfluxDB, Nginx, eCharts, React, web sockets, Docker… that you’ll need to master when building a bot platform.
  • Commercial-grade quality involves spending more and more time on “boring” tasks as you need to polish all aspects of the tool (interface, docs,…) that you can easily skip when just building a prototype.
  • Common challenges to any OSS project like attracting contributors, giving support to users that don’t even say thank you or the cost of maintaining external contributions (e.g. connectors) especially when the core is still evolving
  • Being pragmatic. Make sure everything you do has a good Return of Investment. Example: we went from having official releases, to core releases to no releases (compile from sources). Also, we don’t support older versions and don’t keep backward compatibility. These are all things that would be nice to have but that we cannot afford.
• Technical challenges
  • Eat your own chatbot food for a better language. Evolving your DSLs is expensive, try to design a variety of bots to make sure your language is expressive enough for your needs (e.g. when we moved to our current state-machine language semantics , we had to reimplement the editor, the examples, the docs,…
  • Combine textual and graphical notations . Textual for writing + Graphical for Reading
  • Do not reinvent the wheel. We came close to reinvent Java! (and moved to Xbase )
  • Think long-term. Research prototypes are pure technical debt
  • Security. Are you protected against DDoS attacks via a bot?. It’s not only a security concern but also an economical one if you’re paying to an external provider (e.g. DialogFlow) to process your input texts. There are also privacy concerns if you keep the session data for further bot monitoring and analysis .
• Natural Language Processing
  • NLP engines don’t speak Catalan. Once you’re out of the mainstream languages (English, Spanish, French,….) you may need to combine several NLP engines to cover well the languages you’re interested in.
  • NLP engines are not bilingual. In some countries, bots should be able to combine different languages in the same conversation
  • Everybody seems interested in having sex with a bot. Trolls are also a huge problem with bots. As a brand you don’t want to risk going viral because your bot gave the wrong response to a Troll question.
  • Reusable intent libraries . You don’t want to define again Greetings , Thank You or Troll intents.
  • Testing chatbots is a largely unexplored area • We use Botium to record conversation s to be used as test cases but need much more (e.g. automatic generation of test cases)
  • More training sentences are not always better. Overall quality may decrease if they start overlapping with sentences for other intents.
• Business
  • Most end-users do not want to define chatbots. It doesn’t matter how nice is your interface. They just want to give you their data and get a bot from you.
  • Find your niche. There are over a hundred bot solutions out there. Choose wisely your domain (e.g. our WordPress plugin ) and your business model. E.g. Consider Selling a service (such as our consulting and managed chatbot hosting services ) and not a tool.
  • Understand what really matters to your future clients . You’ll be surprised. For instance, making sure the color of your chatbot widget perfectly matches the look and feel of their website may be more important to them that your latest cool AI-based feature.
  • Help your clients understand what their visitors want to use the bot for. You think you know what your visitors will want to use the bot for but in reality, you don’t. So having a dashboard to monitor the conversations the bot doesn’t know how to answer is vital.
  • Self-learning bots . Your clients will want that but it’s risky. You don’t want the bot to learn the wrong things.

At the end of the talk I also emphasized why we belive you should consider using Xatkit

• As a bot teaching platform. If you teach any course involving bots, chatbots, NLP or DSLs / software modeling and are looking for a use-case or a ready-to-use bot platform for experimentation we’ll be happy to help with materials and assistance and even (remote) seminars
• As a bot research platform . In many bot-related research, researchers spend a significant amount of time putting together the bot pieces they need to create their bots and/or run their experiments. We ask you not to reinvent the wheel and, instead, adopt Xatkit as base platform. You’ll benefit from all Xatkit features and connectors to have powerful bots with less effort and will be able to spend more time on the core research aspects

Lessons learned from building a commercial bot development platform from Jordi Cabot

Of course, mine was just one of the many interesting talks that took place in the workshop. In the BotSE website, you have the full program and links to the videos and slides of all presentations (e.g. don’t miss the great Probot talk by Gregor Martynus ). Thanks again to Emad Shihab , Stefan Wagner and Marco A. Gerosa for inviting me!.

As always, any feedback is appreciated! If you have questions or comments on how to build bots or bot development platforms, share them below!