Documentation

MBot

A modular bot framework.

Home Page (documentation): https://liciojr.github.io/mbot

Git Repository: https://github.com/liciojr/mbot

NPM Package: https://www.npmjs.com/package/mbot

Setup

Quick Start

  1. Clone the repository;
  2. Install dependencies: npm i
  3. Start with adapters/console (*): npm start

(*) other adapters (see "Adapters" section for custom configuration):

  • adapters/rest: npm run start-rest
    • see adapter key configuration in config/core/brain.rest.json
  • adapters/xmpp: npm run start-xmpp
    • see adapter key configuration in config/core/brain.xmpp.json
    • require a XMPP server (e.g. Openfire)

New Project

  1. Create a project:
    • npm init -y;
  2. Install library and dependencies:
    • npm i --save mbot
  3. Create json configuration files in /config (default 'dev' environment);
  4. Create javascript modules in /bot_modules and /kb;
  5. Create app.js with:
    • require('mbot').init();
  6. Run:
    • node app.js

Features

  • modular structure (e.g. communication, knowledge base, classifier, utilities), facilitating collaborative working;
  • centralized multi-environment configuration;
  • flexible integrated logging;
  • universal database abstraction (Sails Waterline) for relational and NoSQL databases;
  • global filesystem overlay;
  • automatic lifecycle management with asynchronous init/dispose;
  • extensible adapters (e.g. console, rest, xmpp, test);
  • helper plugins (e.g. logger, HTML parser/scraper, spreadsheet reader, test stub, HTTP/HTTPS consumer).

Configuration

  • plain JSON format;
  • standard path convention (/config/dir/name.{env}.json);
  • extensible configuration, with MBOT_NODE_ENV and MBOT_NODE_ENV_ALT environment variables (allow partial redefinition, with ALT fallback);
  • built-in environments:
    • dev: development (default)
    • tmp: temporary (git ignored)
    • tst: test automation (with nlp/hashtag)
    • tst_bayes: test automation (with nlp/bayes)
    • rest: rest adapter
    • xmpp: xmpp adapter
    • template: custom template
    • prd: production (suggested)
  • automatic init setup with config modules property;
  • system variable support in any value, replaced at runtime in mbot.config loader:
    {
  "key": "${ENV_VAR_X}"
}

Logging

  • 'util/logger' helper with integrated configuration;
  • shared use with core mbot.logger();
  • allow multiple transport channels and categories.

Database Abstraction

  • 'util/db' helper for direct Waterline integration (without Sails);
  • multiple relational (MySQL, PostgreSQL, FileMaker, Apache Derby, ...) and NoSQL (MongoDB, Redis, OrientDB, ...) database adapters, with same interface;
  • multiple source model integration;
  • allow custom adapters creation;
  • built-in (zero configuration) disk/memory adapter for fast prototyping and test automation.

Overlay and Lifecycle Management

  • core function mbot.load uses filesystem overlay (mbot.path) to import (require) a module, and save its init/dispose callback (if exists);
  • async single/multiple initialization, to instant or previous loaded modules (mbot.init);
  • async global dispose (mbot.dispose).

Adapters

Modules in /adapters folder, with init and onEvent (observer pattern) functions, allowing easily creation of custom adapters.

Choose and configure in config/core/brain.{env}.json file, at adapter.type key.

Built-in adapters:

  • console.js - command-line interface adapter, configuration example:
      "adapter": {
      "type": "console",
      "from": "admin@localhost",
      "prompt": "[question] "
  }
  • xmpp.js - XMPP protocol adapter, configuration example:
      "adapter": {
      "type": "xmpp",
      "setPresence": {
          "status": "chat",
          "text": "alpha version"
      },
      "settings": {
          "jid": "mbot@localhost",
          "password": "${MBOT_XMPP_PWD}",
          "host": "localhost",
          "port": 5222            
      }
  }
  • rest.js - REST service adapter, configuration example:
      "adapter": {
      "type": "rest",
      "get": "/mbot",
      "port": 8080,
      "headers": {
          "Access-Control-Allow-Origin": "*"
      }
  }
  • test.js - Test automation adapter, configuration example:
      "adapter": {
      "type": "test",
      "from": "usr1"
  }

Architecture

The solution has the following structure:

block_diagram

  • config: central configuration directory ({module}.{environment}.json format);
  • adapters: front-end modules (e.g. XMPP, REST, Console);
  • bot_modules: communication modules (specialized dialogs);
  • kb: knowledge base;
  • lib/core: asynchronous core and subsystems facade;
  • nlp: natural language processing modules;
  • util: utilities (e.g. HTML extractor, spreadsheet reader, HTTP consumer, database integration, logger).

Applications should focus on bot_modules and kb sections.

Workflow Summary

A main controller (lib/core/brain) execute the following tasks:

  1. read config/core/brain.{env}.json configuration;
  2. load model and open database configured in db key;
  3. invoke core loader (lib/index) to fetch configured modules in loader.bot_modules key from config/index.{env}.json;
  4. load language module configured in nlp key;
  5. invoke core asynchronous global init;
  6. load and start adapter configured in adapter key;
  7. register adapter onReply event and wait for requests.

For each request, the main controller creates a dialog object with some properties:

{
    from: 'user abc',
    text: 'question xyz',
    state: {},
    action: '',
    entities: [],
    nonEntities: []
}

The NLP module fills entities/nonEntities fields information, and returns a list of compatible intents (communication module / action). By default, following rules apply:

  • if none or 3+ modules are compatible, will return brain.message.unknown configured message;
  • if 2 modules/actions are compatible, will return brain.message.refine and respective help text, for user refinement;
  • if only one module/action is compatible, it will be invoked to format a reply.

A communication module can use the state dictionary to hold user/session information (see bot_modules/examples/echo_bot).

Collaborative Work

The framework was designed to allow multiple concurrent development on the same repository. To accomplish this, follow these steps:

  • define a user tag, example: u01;
  • clone bot_modules/templates/hello_bot to bot_modules/u01/hello_bot, replacing "template" with "u01" in keywords array:
    module.exports = {
  keywords: ['u01'],
  reply
}
  • clone config/index.template.json to config/index.u01.json, replacing "templates" with "u01" in "loader.bot_modules" section:
    {
  "loader": {
      "bot_modules": {
          "u01": "*"
      },
  • define environment variable MBOT_NODE_ENV with u01, and start MBOT;
  • type "u01" to receive a welcome message;
  • to customize main controller features, clone config/core/brain.template.json to config/core/brain.u01.json, replacing whatever you want;
  • any Knowledge Base modules in kb/* can be used, or a new directory kb/u01 can be created to add new ones.

Bot Modules

NLP Modules

Configure a Natural Language Processing module with nlp.type key in config/core/brain.{env}.json file, example:

    "nlp": {
        "type": "bayes",
        "cache": ".tmp/bayes_cache.json",
        "stemmer": "node_modules/natural/lib/natural/stemmers/porter_stemmer.js"
    }

Built-in modules:

  • nlp/hashtag: deterministic classifier, based on rules;
  • nlp/bayes: natural classifier, with Naive-Bayes algorithm.

To create a NLP module, implement a classify function in nlp folder, with 2 arguments: a dialog and a callback. Remember to configure nlp.type key.

Current Named Entity Recognition use a simplified rule set:

  • text between quotes, example:
    • how old is "Peter Parker"?
      • entities ["Peter Parker"]
  • words starting with digit, example:
    • sum 5 and 6
      • entities ["5", "6"]
  • words starting with uppercase, except at the beginning of a sentence, and with more than one letter, example:
    • What is Peter phone?
      • entities ["Peter"]

Communication Modules

Files under bot_modules folder, with following interface:

  • Required:
    • keywords property (string array): first element required (master keyword for deterministic classifier). Other keywords used for natural language classifier;
    • reply(dialog, callback) function;
  • Optional:
    • actions property (object): action dictionary, each entry with keywords string/array;
    • help property (string): help text.

Examples:

  • currency_bot.js - display a list of countries related to a currency symbol, example:
    • which countries use "GBP" currency?
    • "BRL" country
  • degree_bot.js - temperature conversion bot, examples:
    • 95 to celsius
    • 35 to fahrenheit
  • echo_bot.js - simple echo message, example:
    • some echo here
  • geo_bot.js - display your/some IP address location, examples:
    • my address?
    • ip of "github.com"
  • google_bot.js - show latest technology news from google site, example:
    • tech news
  • hobby_bot.js - user hobby list manager, examples:
    • please, include a new hobby for me: Carpentry
    • show my hobbies

Knowledge Base Modules

Any file under kb folder. Examples:

  • currency_kb.js - use util/spreadsheet helper to read information from a spreadsheet file;
  • degree_kb.js - use soap external library to consume a public SOAP webservice at webservicex.net;
  • geo_kb.js - use util/http helper to consume a public REST service at http://freegeoip.net;
  • google_kb.js - use util/html helper to extract information from a Google HTML page;
  • hobby_kb.js - use util/db helper to interact with a database.

Tests

First time, type npm install to fetch dependencies, then type npm test.

Test support:

  • adapters/test.js: expose "request" method for test automation;
  • util/test.js: utility helper for test implementation, compatible with Mocha framework;
  • test/examples/*: implementation examples.

License

GNU Lesser General Public License (LGPL), version 3 or later.

See the LICENSE file in the root directory or https://www.gnu.org/licenses/lgpl.html.