Database Adapters (github.com/moleculer-go/store)

---

Moleculer framework has an official set of Database adapters. Use them to persist your data in a database.

{% note info Database per service%} Moleculer follows the one database per service pattern. To learn more about this design pattern and its implications check this article. {% endnote %}

Features

• default CRUD actions (create, find, count, list, get, update, remove)
• cached actions
• pagination support
• pluggable adapter - There is the default memory adapter for testing & prototyping)
• official adapters for MongoDB.
• fields filtering
• populating
• encode/decode IDs
• entity lifecycle events for notifications

Memory Adapter

Moleculer's memory adapter uses hashicorp/go-memdb. Use it to quickly set up and test you prototype and for writing test cases.

{% note warn%} Only use this adapter for prototyping and testing. When you are ready to go into production simply swap to Mongo ... adapters as they all implement common Settings, Actions. {% endnote %}

Install

$ go get -u github.com/moleculer-go/store

# Or with go modules
github.com/moleculer-go/store v1.0.1

Usage

package main

import (
  "fmt"
  "time"

  "github.com/moleculer-go/store"

  "github.com/moleculer-go/moleculer"
  "github.com/moleculer-go/moleculer/broker"
)

func main() {
  var bkr = broker.New(&moleculer.Config{LogLevel: "info"})
  bkr.Publish(moleculer.ServiceSchema{
    Name: "users",
    Settings: map[string]interface{}{
      "fields":    []string{"_id", "username", "name"},
      "populates": map[string]interface{}{"friends": "users.get"},
    },
    Mixins: []moleculer.Mixin{store.Mixin(&store.MemoryAdapter{
      Table:        "users",
      SearchFields: []string{"name", "username"},
    })},
  })
  bkr.Start()
  time.Sleep(time.Millisecond * 300)
  user := <-bkr.Call("users.create", map[string]interface{}{
    "username": "john",
    "name":     "John Doe",
    "status":   1,
  })

  id := user.Get("id").String()
  // Get all users
  fmt.Printf("all users: ", <-bkr.Call("users.find", map[string]interface{}{}))

  // List users with pagination
  fmt.Printf("list users: ", <-bkr.Call("users.list", map[string]interface{}{
    "page":     2,
    "pageSize": 10,
  }))

  idParam := map[string]interface{}{"id": id}

  // Get a user
  fmt.Printf("get user: ", <-bkr.Call("users.get", idParam))

  // Update a user
  fmt.Printf("update user: ", <-bkr.Call("users.update", map[string]interface{}{
    "id":   id,
    "name": "Jane Doe",
  }))

  // Print user after update
  fmt.Printf("get user: ", <-bkr.Call("users.get", idParam))

  // Delete a user
  fmt.Printf("remove user: ", <-bkr.Call("users.remove", idParam))

  bkr.Stop()
}

run the example above with:

$ go run github.com/moleculer-go/store/examples/users

More examples can be found on GitHub

Settings

All DB adapters share these common set of settings:

Property	Type	Default	Description
idField	string	required	Name of ID field.
fields	[]string	["**"]	Field filtering list. It must be an Array. If the value is nil it will assume ["**"] and it will not filter the fields of entities.
populates	map[string]interface{}		Schema for population. Read more.
pageSize	Number	required	Default page size in list action.
maxPageSize	Number	required	Maximum page size in list action.
maxLimit	Number	required	Maximum value of limit in find action. Default: -1 (no limit)
entityValidator	Object, function	null	Validator schema or a function to validate the incoming entity in create action.

Actions

DB adapters also implement CRUD operations. These actions are public methods and can be called by other services.

find

Find entities by query.

Parameters

Property	Type	Default	Description
populate	[]string	-	Populated fields.
fields	[]string	-	Fields filter.
limit	Number	required	Max count of rows.
offset	Number	required	Count of skipped rows.
sort	string	required	Sorted fields.
search	string	required	Search text.
searchFields	string	required	Fields for searching.
query	map[string]interface{}	required	Query object. Passes to adapter.

Results

Type: moluculer.Paylod - List of found entities.

count

Get count of entities by query.

Parameters

Property	Type	Default	Description
search	string	required	Search text.
searchFields	string	required	Fields list for searching.
query	Object	required	Query object. Passes to adapter.

Results

Type: Number - Count of found entities.

list

List entities by filters and pagination results.

Parameters

Property	Type	Default	Description
populate	[]string	-	Populated fields.
fields	[]string	-	Fields filter.
page	Number	required	Page number.
pageSize	Number	required	Size of a page.
sort	string	required	Sorted fields.
search	string	required	Search text.
searchFields	string	required	Fields for searching.
query	map[string]interface{}	required	Query object. Passes to adapter.

Results

Type: moleculer.Payload - List of found entities and count.

create

Create a new entity.

Parameters

Payload with fields to be saved in the new entity record.

Results

Type: moleculer.Payload - Saved entity.

get

Get entity by ID.

Parameters

Property	Type	Default	Description
id	string	required	ID of entity.
ids	[]string	required	ID(s) of entities.
populate	[]string	-	Field list for populate.
fields	[]string	-	Fields filter.
mapping	Bool	-	Convert the returned Array to Map where the key is the value of id.

Results

Type: moleculer.Payload - Found entity(ies).

update

Update an entity by ID.

After update, clear the cache & call lifecycle events.

Parameters

Property	Type	Default	Description
id	string	-	Id of the records being updated.

Results

Type: moleculer.Payload - Updated entity.

remove

Remove an entity by ID.

Parameters

Property	Type	Default	Description
id	string	required	ID of entity.

Results

Type: Number - Count of removed entities.

Populating

The service allows you to easily populate fields from other services. For exapmle: If you have an author field in post entity, you can populate it with users service by ID of author. If the field is an Array of IDs, it will populate all entities via only one request

Example of populate schema

package main

import (
 "fmt"
 "time"

 "github.com/moleculer-go/store"

 "github.com/moleculer-go/moleculer"
 "github.com/moleculer-go/moleculer/broker"
)

func main() {
 var bkr = broker.New(&moleculer.Config{LogLevel: "info"})
 bkr.Publish(moleculer.Service{
  Name: "users",
  Settings: map[string]interface{}{
   "fields":    []string{"id", "username", "name"},
   "populates": map[string]interface{}{"friends": "users.get"},
  },
  Mixins: []moleculer.Mixin{store.Mixin(&store.MemoryAdapter{
   Table:        "users",
   SearchFields: []string{"name", "username"},
  })},
 })
 bkr.Publish(moleculer.Service{
  Name: "posts",
  Settings: map[string]interface{}{
   "populates": map[string]interface{}{
    //Shorthand populate rule. Resolve the 'voters' values with the users.get action.
    "voters": "users.get",
    // Define the params of action call.
    //It will receive only with username of author.
    "author": map[string]interface{}{
     "action": "users.get",
     "params": map[string]interface{}{
      "fields": []string{"username"},
     },
    },
   },
  },
  Mixins: []moleculer.Mixin{store.Mixin(&store.MemoryAdapter{
   Table: "posts",
  })},
 })
 bkr.Start()
 time.Sleep(time.Millisecond * 300)

 johnSnow := <-bkr.Call("users.create", map[string]interface{}{
  "name":     "John",
  "lastname": "Snow",
  "username": "jsnow",
  "fullname": "John Snow",
 })
 marie := <-bkr.Call("users.create", map[string]interface{}{
  "name":     "Marie",
  "lastname": "Claire",
  "username": "mclaire",
  "fullname": "Marie Claire",
 })

 post := <-bkr.Call("posts.create", map[string]interface{}{
  "content": "Lorem ipsum dolor sit amet, consectetur ...",
  "voters":  []string{marie.Get("id").String()},
  "author":  johnSnow.Get("id").String(),
  "status":  1,
 })

 // List posts with populated author
 fmt.Printf("posts with author: ", <-bkr.Call("posts.find", map[string]interface{}{
  "populate": []string{"author"},
 }))

 // List posts with populated voters
 fmt.Printf("posts with voters: ", <-bkr.Call("posts.find", map[string]interface{}{
  "populate": []string{"voters"},
 }))

 // remove post
 <-bkr.Call("posts.remove", map[string]interface{}{
  "id": post.Get("id").String(),
 })

 //remove users
 <-bkr.Call("users.remove", map[string]interface{}{
  "id": johnSnow.Get("id").String(),
 })
 <-bkr.Call("users.remove", map[string]interface{}{
  "id": marie.Get("id").String(),
 })

 bkr.Stop()
}

run the example above with:
$ go run github.com/moleculer-go/store/examples/populates

The populate parameter is available in find, list and get actions.

Extend with custom actions

Naturally you can extend this service with your custom actions.

package main

import (
 "fmt"
 "time"

 "github.com/moleculer-go/store"

 "github.com/moleculer-go/moleculer"
 "github.com/moleculer-go/moleculer/broker"
)

func main() {
 var bkr = broker.New(&moleculer.Config{LogLevel: "info"})
 bkr.Publish(moleculer.Service{
  Name: "users",
  Settings: map[string]interface{}{
   "fields":    []string{"id", "username", "name"},
   "populates": map[string]interface{}{"friends": "users.get"},
  },
  Mixins: []moleculer.Mixin{store.Mixin(&store.MemoryAdapter{
   Table:        "users",
   SearchFields: []string{"name", "username"},
  })},
 })
 adapter := &store.MemoryAdapter{
  Table: "posts",
 }
 bkr.Publish(moleculer.Service{
  Name: "posts",
  Settings: map[string]interface{}{
   "populates": map[string]interface{}{
    //Shorthand populate rule. Resolve the 'voters' values with the users.get action.
    "voters": "users.get",
    // Define the params of action call. It will receive only with username & full name of author.
    "author": map[string]interface{}{
     "action": "users.get",
     "params": map[string]interface{}{
      "fields": []string{"username", "fullname"},
     },
    },
   },
  },
  Mixins: []moleculer.Mixin{store.Mixin(adapter)},
  Actions: []moleculer.Action{
   {
    Name: "byAuthors",
    Handler: func(ctx moleculer.Context, params moleculer.Payload) interface{} {
     return <-ctx.Call("posts.find", map[string]interface{}{
      "query": map[string]interface{}{
       "author": params.Get("authorId").String(),
      },
      "limit": 10,
      "sort":  "-createdAt",
     })
    },
   },
  },
 })
 bkr.Start()
 time.Sleep(time.Millisecond * 300)

 johnSnow := <-bkr.Call("users.create", map[string]interface{}{
  "name":     "John",
  "lastname": "Snow",
  "username": "jsnow",
  "fullname": "John Snow",
 })
 marie := <-bkr.Call("users.create", map[string]interface{}{
  "name":     "Marie",
  "lastname": "Claire",
  "username": "mclaire",
  "fullname": "Marie Claire",
 })

 post := <-bkr.Call("posts.create", map[string]interface{}{
  "content": "Lorem ipsum dolor sit amet, consectetur ...",
  "voters":  []string{marie.Get("id").String()},
  "author":  johnSnow.Get("id").String(),
  "status":  1,
 })

 // List posts with populated authors
 fmt.Printf("posts by authors: ", <-bkr.Call("posts.byAuthors", map[string]interface{}{
  "authorId": johnSnow.Get("id").String(),
 }))

 // remove post
 <-bkr.Call("posts.remove", map[string]interface{}{
  "id": post.Get("id").String(),
 })

 //remove users
 <-bkr.Call("users.remove", map[string]interface{}{
  "id": johnSnow.Get("id").String(),
 })
 <-bkr.Call("users.remove", map[string]interface{}{
  "id": marie.Get("id").String(),
 })

 bkr.Stop()
}

run the example above with:
$ go run github.com/moleculer-go/store/examples/customActions

Cache

Not Implemented yet!

Mongo Adapter

This adapter is based on MongoDB.

Install

$ go get -u github.com/moleculer-go/store

# Or with go modules
github.com/moleculer-go/store v1.0.1

{% note info Dependencies%} To use this adapter you need to install MongoDB on you system. {% endnote %}

Usage

package main

import (
  "fmt"
  "time"

  "github.com/moleculer-go/store"
  "github.com/moleculer-go/store/mongo"
  "github.com/spf13/cobra"

  "github.com/moleculer-go/moleculer"
  "github.com/moleculer-go/moleculer/broker"
  "github.com/moleculer-go/moleculer/cli"
)

func main() {
  cli.Start(
    &moleculer.Config{LogLevel: "info"},
    func(broker *broker.ServiceBroker, cmd *cobra.Command) {
      broker.Publish(moleculer.ServiceSchema{
        Name: "users",
        Settings: map[string]interface{}{
          "fields":    []string{"_id", "username", "name"},
          "populates": map[string]interface{}{"friends": "users.get"},
        },
        Mixins: []moleculer.Mixin{store.Mixin(&mongo.MongoAdapter{
          MongoURL:   "mongodb://localhost:27017",
          Collection: "users",
          Database:   "test",
          Timeout:    time.Second * 5,
        })},
      })
      broker.Start()
      time.Sleep(time.Millisecond * 300)
      user := <-broker.Call("users.create", map[string]interface{}{
        "username": "john",
        "name":     "John Doe",
        "status":   1,
      })

      id := user.Get("id").String()
      // Get all users
      fmt.Printf("all users: ", <-broker.Call("users.find", map[string]interface{}{}))

      // List users with pagination
      fmt.Printf("list users: ", <-broker.Call("users.list", map[string]interface{}{
        "page":     2,
        "pageSize": 10,
      }))

      idParam := map[string]interface{}{"id": id}

      // Get a user
      fmt.Printf("get user: ", <-broker.Call("users.get", idParam))

      // Update a user
      fmt.Printf("update user: ", <-broker.Call("users.update", map[string]interface{}{
        "id":   id,
        "name": "Jane Doe",
      }))

      // Print user after update
      fmt.Printf("get user: ", <-broker.Call("users.get", idParam))

      // Delete a user
      fmt.Printf("remove user: ", <-broker.Call("users.remove", idParam))

      broker.Stop()

    })
}

run the example above with:

$ go run github.com/moleculer-go/store/examples/usersMongo start

More MongoDB examples can be found on GitHub

SQLite Adapter

This adapter is based on crawshaw/sqlite.

Install

$ go get -u github.com/moleculer-go/store

# Or with go modules
github.com/moleculer-go/store v1.0.1

Usage

package main

import (
  "fmt"
  "time"

  "github.com/moleculer-go/store"
  "github.com/moleculer-go/store/sqlite"
  "github.com/spf13/cobra"

  "github.com/moleculer-go/moleculer"
  "github.com/moleculer-go/moleculer/broker"
  "github.com/moleculer-go/moleculer/cli"
)

func main() {
  cli.Start(
    &moleculer.Config{LogLevel: "debug"},
    func(broker *broker.ServiceBroker, cmd *cobra.Command) {
      broker.Publish(moleculer.ServiceSchema{
        Name: "users",
        Settings: map[string]interface{}{
          "fields":    []string{"id", "username", "name"},
          "populates": map[string]interface{}{"friends": "users.get"},
        },
        Mixins: []moleculer.Mixin{store.Mixin(&sqlite.Adapter{
          URI:   "file:memory:?mode=memory",
          Table: "users",
          Columns: []sqlite.Column{
            {
              Name: "username",
              Type: "string",
            },
            {
              Name: "name",
              Type: "string",
            },
            {
              Name: "status",
              Type: "integer",
            },
          },
        })},
      })
      broker.Start()
      time.Sleep(time.Millisecond * 300)
      user := <-broker.Call("users.create", map[string]interface{}{
        "username": "john",
        "name":     "John Doe",
        "status":   1,
      })

      id := user.Get("id").String()
      // Get all users
      fmt.Printf("all users: ", <-broker.Call("users.find", map[string]interface{}{}))

      // List users with pagination
      fmt.Printf("list users: ", <-broker.Call("users.list", map[string]interface{}{
        "page":     2,
        "pageSize": 10,
      }))

      idParam := map[string]interface{}{"id": id}

      // Get a user
      fmt.Printf("get user: ", <-broker.Call("users.get", idParam))

      // Update a user
      fmt.Printf("update user: ", <-broker.Call("users.update", map[string]interface{}{
        "id":   id,
        "name": "Jane Doe",
      }))

      // Print user after update
      fmt.Printf("get user: ", <-broker.Call("users.get", idParam))

      // Delete a user
      fmt.Printf("remove user: ", <-broker.Call("users.remove", idParam))

      broker.Stop()

    })
}

run the example above with:

$ go run github.com/moleculer-go/store/examples/usersSQLite start

More Database adaptor examples can be found on GitHub