YUKI Hiroshi
null+****@clear*****
Wed Feb 26 17:20:51 JST 2014
YUKI Hiroshi 2014-02-26 17:20:51 +0900 (Wed, 26 Feb 2014) New Revision: 23205147d7e9b52737e0de2200574f7d2d861dd7 https://github.com/droonga/droonga.org/commit/23205147d7e9b52737e0de2200574f7d2d861dd7 Message: Rewrite tutorial of handler plugin (WIP) Modified files: tutorial/plugin-development/handler/index.md Modified: tutorial/plugin-development/handler/index.md (+91 -87) =================================================================== --- tutorial/plugin-development/handler/index.md 2014-02-26 16:07:17 +0900 (a6088b3) +++ tutorial/plugin-development/handler/index.md 2014-02-26 17:20:51 +0900 (5c267ff) @@ -10,56 +10,67 @@ layout: en ## The goal of this tutorial -This tutorial aims to help you to learn how to develop plugins -which extends operations in handle phrase. +This tutorial aims to help you to learn how to develop plugins which do something dispersively for/in each partition, around the handling phase. ## Precondition -* You must complete [Modify requests and responses tutorial][adapter]. +* You must complete the [tutorial for the adaption phase][adapter]. -## Handling phase +## Handling of incoming messages -The handling phase is the phase that the actual storage access is happen. -As Droonga is a distributed system, handler phase is done in multiple partitions. +When an incoming message is transferred from the adaption phase, the Droonga Engine plans how distribute it to multiple partitions. +Then *each partition simply processes only one distributed message as its input, and returns a result.* +After that, the Droonga Engine collects results from partitions and transfer the unified result to the post adaption phase. -Here, in this tutorial, we are going to replace the handling phase of `search` command for explanation. This breaks the `search` command. So this is not useful in practice, but it will help you to learn how Droonga works. +That operation in each partition is called *handling phase*. +A class to define operations at the phase is called *handler*. +Adding of a new handler means adding a new command. -In practice, we need to *extend* Droonga. In this case, we need to add a new command which does not conflict with the existing commands. To do so, you need to learn not only how to handle messages but also how to distribute messages to handlers and collect messages from them. Proceed to [Distribute requests and collect responses][] after this tutorial completed. +The handling phase is the time that actual storage accesses happen. +Actually, handlers for some commands (`search`, `add`, `create_table` and so on) access to the storage at the time. -TODO fix the link to "Distribute requests and collect responses" tutorial -## Directory Structure +## Read-only handler -The directory structure for plugins are in same rule as explained in [Modify requests and responses tutorial][adapter]. +Here, in this tutorial, we are going to add a new handler for our custom `countRecords` command. +The command reports the number of records about a specified table, for each partition. +So this command helps you to know how records are distributed in the cluster. -Now let's create `sample-logger` plugin again. This will act almost same as [Modify requests and responses tutorial][adapter] version, except the phase in which the plugin works. We need to put `sample-logger.rb` to `lib/droonga/plugins/sample-logger.rb`. The directory tree will be like this: +The command doesn't change the storage, so it is *read-only* handler. + +### Directory Structure + +The directory structure for plugins are in same rule as explained in the [tutorial for the adaption phase][adapter]. +Now let's create the `count-records` plugin, as the file `count-records.rb`. The directory tree will be: ~~~ lib └── droonga └── plugins - └── sample-logger.rb + └── count-records.rb ~~~ -## Create a plugin +### Create a plugin Create a plugin as follows: -lib/droonga/plugins/sample-logger.rb: +lib/droonga/plugins/count-records.rb: ~~~ruby require "droonga/plugin" module Droonga module Plugins - module SampleLoggerPlugin - Plugin.registry.register("sample-logger", self) + module CountRecordsPlugin + Plugin.registry.register("count-records", self) class Handler < Droonga::Handler - message.type = "search" + # Declare that this handles incoming messages with the type "countRecords". + message.type = "countRecords" def handle(message, messenger) - $log.info "Droonga::Plugins::SampleLoggerPlugin", :message => message + # The returned value of this method will become the body of the response. + { "count": [0] } end end end @@ -67,120 +78,113 @@ module Droonga end ~~~ -## Activate the plugin with `catalog.json` +### Activate the plugin with `catalog.json` -Update catalog.json to activate this plugin. Add `"sample-logger"` to `"plugins"`. +Update catalog.json to activate this plugin. Add `"count-records"` to `"plugins"`. ~~~ (snip) "datasets": { "Starbucks": { (snip) - "plugins": ["sample-logger", "groonga", "crud", "search"], + "plugins": ["count-records", "groonga", "crud", "search"], (snip) ~~~ -## Run +### Run -Let's get Droonga started. Note that you need to specify ./lib directory in RUBYLIB environment variable in order to make ruby possible to find your plugin. +Let's get Droonga started. +Note that you need to specify ./lib directory in RUBYLIB environment variable in order to make ruby possible to find your plugin. # kill $(cat fluentd.pid) # RUBYLIB=./lib fluentd --config fluentd.conf --log fluentd.log --daemon fluentd.pid -## Test +### Test -Send a search request to Droonga Engine. Use `search-columbus.json` same as of [Modify requests and responses tutorial][adapter]. +Send a message for the `countRecords` command to the Droonga Engine. +First, create a JSON file for the message as `count-records.json`, like: -~~~ -# droonga-request --tag starbucks search-columbus.json -~~~ +count-records.json: -You will see no output for `droonga-request` execution because out `sample-logger` plugin traps the `add` request. +~~~json +{ + "dataset" : "Starbucks", + "type" : "countRecords", + "body" : {} +} +~~~ -Instead, you will see something like these lines in `fluentd.log`: +The message body is blank for now. +OK, let's send it. ~~~ -2014-02-17 16:25:23 +0900 [info]: Droonga::Plugins::SampleLoggerPlugin message=#<Droonga::HandlerMessage:0x007f9a7f0987a8 @raw={"dataset"=>"Starbucks", "type"=>"search", "body"=>{"id"=>"localhost:24224/starbucks.#0", "task"=>{"route"=>"localhost:24224/starbucks.011", "step"=>{"command"=>"search", "dataset"=>"Starbucks", "body"=>{"queries"=>{"stores"=>{"source"=>"Store", "condition"=>{"query"=>"Columbus", "matchTo"=>"_key"}, "output"=>{"elements"=>["startTime", "elapsedTime", "count", "attributes", "records"], "attributes"=>["_key"], "limit"=>-1}}}}, "type"=>"broadcast", "outputs"=>["errors", "stores"], "replica"=>"random", "routes"=>["localhost:24224/starbucks.001", "localhost:24224/starbucks.011", "localhost:24224/starbucks.020"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#0"], "stores"=>["localhost:24224/starbucks.#0"]}}, "n_of_inputs"=>0, "values"=>{}}, "descendants"=>{"errors"=>["localhost:24224/starbucks"], "stores"=>["localhost:24224/star bucks"]} }, "replyTo"=>{"type"=>"search.result", "to"=>"127.0.0.1:50410/droonga"}, "id"=>"1392621923.903868", "date"=>"2014-02-17 16:25:23 +0900", "appliedAdapters"=>["Droonga::Plugins::Error::Adapter"]}, @body={"id"=>"localhost:24224/starbucks.#0", "task"=>{"route"=>"localhost:24224/starbucks.011", "step"=>{"command"=>"search", "dataset"=>"Starbucks", "body"=>{"queries"=>{"stores"=>{"source"=>"Store", "condition"=>{"query"=>"Columbus", "matchTo"=>"_key"}, "output"=>{"elements"=>["startTime", "elapsedTime", "count", "attributes", "records"], "attributes"=>["_key"], "limit"=>-1}}}}, "type"=>"broadcast", "outputs"=>["errors", "stores"], "replica"=>"random", "routes"=>["localhost:24224/starbucks.001", "localhost:24224/starbucks.011", "localhost:24224/starbucks.020"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#0"], "stores"=>["localhost:24224/starbucks.#0"]}}, "n_of_inputs"=>0, "values"=>{}}, "descendants"=>{"errors"=>["localhost:24224/starbucks"], "stores" =>["loca lhost:24224/starbucks"]}}, @task={"route"=>"localhost:24224/starbucks.011", "step"=>{"command"=>"search", "dataset"=>"Starbucks", "body"=>{"queries"=>{"stores"=>{"source"=>"Store", "condition"=>{"query"=>"Columbus", "matchTo"=>"_key"}, "output"=>{"elements"=>["startTime", "elapsedTime", "count", "attributes", "records"], "attributes"=>["_key"], "limit"=>-1}}}}, "type"=>"broadcast", "outputs"=>["errors", "stores"], "replica"=>"random", "routes"=>["localhost:24224/starbucks.001", "localhost:24224/starbucks.011", "localhost:24224/starbucks.020"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#0"], "stores"=>["localhost:24224/starbucks.#0"]}}, "n_of_inputs"=>0, "values"=>{}}, @step={"command"=>"search", "dataset"=>"Starbucks", "body"=>{"queries"=>{"stores"=>{"source"=>"Store", "condition"=>{"query"=>"Columbus", "matchTo"=>"_key"}, "output"=>{"elements"=>["startTime", "elapsedTime", "count", "attributes", "records"], "attributes"=>["_key"], "limit"=>-1} }}}, "ty pe"=>"broadcast", "outputs"=>["errors", "stores"], "replica"=>"random", "routes"=>["localhost:24224/starbucks.001", "localhost:24224/starbucks.011", "localhost:24224/starbucks.020"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#0"], "stores"=>["localhost:24224/starbucks.#0"]}}> -2014-02-17 16:25:23 +0900 [info]: Droonga::Plugins::SampleLoggerPlugin message=#<Droonga::HandlerMessage:0x007f9a7f060970 @raw={"dataset"=>"Starbucks", "type"=>"search", "body"=>{"id"=>"localhost:24224/starbucks.#0", "task"=>{"route"=>"localhost:24224/starbucks.020", "step"=>{"command"=>"search", "dataset"=>"Starbucks", "body"=>{"queries"=>{"stores"=>{"source"=>"Store", "condition"=>{"query"=>"Columbus", "matchTo"=>"_key"}, "output"=>{"elements"=>["startTime", "elapsedTime", "count", "attributes", "records"], "attributes"=>["_key"], "limit"=>-1}}}}, "type"=>"broadcast", "outputs"=>["errors", "stores"], "replica"=>"random", "routes"=>["localhost:24224/starbucks.001", "localhost:24224/starbucks.011", "localhost:24224/starbucks.020"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#0"], "stores"=>["localhost:24224/starbucks.#0"]}}, "n_of_inputs"=>0, "values"=>{}}, "descendants"=>{"errors"=>["localhost:24224/starbucks"], "stores"=>["localhost:24224/star bucks"]} }, "replyTo"=>{"type"=>"search.result", "to"=>"127.0.0.1:50410/droonga"}, "id"=>"1392621923.903868", "date"=>"2014-02-17 16:25:23 +0900", "appliedAdapters"=>["Droonga::Plugins::Error::Adapter"]}, @body={"id"=>"localhost:24224/starbucks.#0", "task"=>{"route"=>"localhost:24224/starbucks.020", "step"=>{"command"=>"search", "dataset"=>"Starbucks", "body"=>{"queries"=>{"stores"=>{"source"=>"Store", "condition"=>{"query"=>"Columbus", "matchTo"=>"_key"}, "output"=>{"elements"=>["startTime", "elapsedTime", "count", "attributes", "records"], "attributes"=>["_key"], "limit"=>-1}}}}, "type"=>"broadcast", "outputs"=>["errors", "stores"], "replica"=>"random", "routes"=>["localhost:24224/starbucks.001", "localhost:24224/starbucks.011", "localhost:24224/starbucks.020"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#0"], "stores"=>["localhost:24224/starbucks.#0"]}}, "n_of_inputs"=>0, "values"=>{}}, "descendants"=>{"errors"=>["localhost:24224/starbucks"], "stores" =>["loca lhost:24224/starbucks"]}}, @task={"route"=>"localhost:24224/starbucks.020", "step"=>{"command"=>"search", "dataset"=>"Starbucks", "body"=>{"queries"=>{"stores"=>{"source"=>"Store", "condition"=>{"query"=>"Columbus", "matchTo"=>"_key"}, "output"=>{"elements"=>["startTime", "elapsedTime", "count", "attributes", "records"], "attributes"=>["_key"], "limit"=>-1}}}}, "type"=>"broadcast", "outputs"=>["errors", "stores"], "replica"=>"random", "routes"=>["localhost:24224/starbucks.001", "localhost:24224/starbucks.011", "localhost:24224/starbucks.020"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#0"], "stores"=>["localhost:24224/starbucks.#0"]}}, "n_of_inputs"=>0, "values"=>{}}, @step={"command"=>"search", "dataset"=>"Starbucks", "body"=>{"queries"=>{"stores"=>{"source"=>"Store", "condition"=>{"query"=>"Columbus", "matchTo"=>"_key"}, "output"=>{"elements"=>["startTime", "elapsedTime", "count", "attributes", "records"], "attributes"=>["_key"], "limit"=>-1} }}}, "ty pe"=>"broadcast", "outputs"=>["errors", "stores"], "replica"=>"random", "routes"=>["localhost:24224/starbucks.001", "localhost:24224/starbucks.011", "localhost:24224/starbucks.020"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#0"], "stores"=>["localhost:24224/starbucks.#0"]}}> -2014-02-17 16:25:23 +0900 [info]: Droonga::Plugins::SampleLoggerPlugin message=#<Droonga::HandlerMessage:0x007f9a7f069c50 @raw={"dataset"=>"Starbucks", "type"=>"search", "body"=>{"id"=>"localhost:24224/starbucks.#0", "task"=>{"route"=>"localhost:24224/starbucks.001", "step"=>{"command"=>"search", "dataset"=>"Starbucks", "body"=>{"queries"=>{"stores"=>{"source"=>"Store", "condition"=>{"query"=>"Columbus", "matchTo"=>"_key"}, "output"=>{"elements"=>["startTime", "elapsedTime", "count", "attributes", "records"], "attributes"=>["_key"], "limit"=>-1}}}}, "type"=>"broadcast", "outputs"=>["errors", "stores"], "replica"=>"random", "routes"=>["localhost:24224/starbucks.001", "localhost:24224/starbucks.011", "localhost:24224/starbucks.020"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#0"], "stores"=>["localhost:24224/starbucks.#0"]}}, "n_of_inputs"=>0, "values"=>{}}, "descendants"=>{"errors"=>["localhost:24224/starbucks"], "stores"=>["localhost:24224/star bucks"]} }, "replyTo"=>{"type"=>"search.result", "to"=>"127.0.0.1:50410/droonga"}, "id"=>"1392621923.903868", "date"=>"2014-02-17 16:25:23 +0900", "appliedAdapters"=>["Droonga::Plugins::Error::Adapter"]}, @body={"id"=>"localhost:24224/starbucks.#0", "task"=>{"route"=>"localhost:24224/starbucks.001", "step"=>{"command"=>"search", "dataset"=>"Starbucks", "body"=>{"queries"=>{"stores"=>{"source"=>"Store", "condition"=>{"query"=>"Columbus", "matchTo"=>"_key"}, "output"=>{"elements"=>["startTime", "elapsedTime", "count", "attributes", "records"], "attributes"=>["_key"], "limit"=>-1}}}}, "type"=>"broadcast", "outputs"=>["errors", "stores"], "replica"=>"random", "routes"=>["localhost:24224/starbucks.001", "localhost:24224/starbucks.011", "localhost:24224/starbucks.020"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#0"], "stores"=>["localhost:24224/starbucks.#0"]}}, "n_of_inputs"=>0, "values"=>{}}, "descendants"=>{"errors"=>["localhost:24224/starbucks"], "stores" =>["loca lhost:24224/starbucks"]}}, @task={"route"=>"localhost:24224/starbucks.001", "step"=>{"command"=>"search", "dataset"=>"Starbucks", "body"=>{"queries"=>{"stores"=>{"source"=>"Store", "condition"=>{"query"=>"Columbus", "matchTo"=>"_key"}, "output"=>{"elements"=>["startTime", "elapsedTime", "count", "attributes", "records"], "attributes"=>["_key"], "limit"=>-1}}}}, "type"=>"broadcast", "outputs"=>["errors", "stores"], "replica"=>"random", "routes"=>["localhost:24224/starbucks.001", "localhost:24224/starbucks.011", "localhost:24224/starbucks.020"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#0"], "stores"=>["localhost:24224/starbucks.#0"]}}, "n_of_inputs"=>0, "values"=>{}}, @step={"command"=>"search", "dataset"=>"Starbucks", "body"=>{"queries"=>{"stores"=>{"source"=>"Store", "condition"=>{"query"=>"Columbus", "matchTo"=>"_key"}, "output"=>{"elements"=>["startTime", "elapsedTime", "count", "attributes", "records"], "attributes"=>["_key"], "limit"=>-1} }}}, "ty pe"=>"broadcast", "outputs"=>["errors", "stores"], "replica"=>"random", "routes"=>["localhost:24224/starbucks.001", "localhost:24224/starbucks.011", "localhost:24224/starbucks.020"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#0"], "stores"=>["localhost:24224/starbucks.#0"]}}> +# droonga-request --tag starbucks count-records.json +Elapsed time: 0.01494 +[ + "droonga.message", + 1392621168, + { + "inReplyTo": "1392621168.0119512", + "statusCode": 200, + "type": "countRecords.result", + "body": { + "count": [0, 0, 0] + } + } +] ~~~ -Note that three lines are shown for only one request. What is happening? +Then you'll get a response message like above. +Look at these points: -Remember that we have configured `Starbucks` dataset to use three partitions (and each has two replicas) in `catalog.json` of [the basic tutorial][basic]. + * The `type` of the response becomes `countRecords.result`. It is automatically named by the Droonga Engine. + * The format of the `body` is same to the returned value of the handler's `handle` method. -The `search` request is dispatched to three partitions and passed into handling phase for each partition. That is because we saw three lines for one request. +However, there are three elements in the `count` array. Why? -The messages shown is in internal format, which is transformed from the request you've sent. -You can see your search request is distributed to partitions `localhost:24224/starbucks.001`, `localhost:24224/starbucks.011` and `localhost:24224/starbucks.020` from `"routes"`. + * Remember that we have configured `Starbucks` dataset to use three partitions (and each has two replicas) in the `catalog.json` of [the basic tutorial][basic]. + * Because it is a read-only handler, the incoming message is distributed only to paritions, not to replicas. + So there are only three results, not six. + * The Droonga Engine automatically collects results from parititions. + Those three results are joined to just one array. -In `search` case, it is enough to use one replica per one partition because replicas for a partition are expected to have the exactly same contents. -So the planner ordered distributor to choose one replica randomly. +(TODO: I have to add a figure to indicate active nodes: [000, 001, 010, 011, 020, 021] => [000, 011, 020]) -## Trap "add" command +As the result, just one array with three elements appears in the response message. -We have seen how distributed search is done from the view point of handling phase so far. -How about `"add"` command? -Update `smaple-logger` plugin to trap `"add"` message instead of `"search"`. +### Design input and output -lib/droonga/plugins/sample-logger.rb: +(TBD) -~~~ -require "droonga/plugin" -module Droonga - module Plugins - module SampleLoggerPlugin - Plugin.registry.register("sample-logger", self) - - class Handler < Droonga::Handler - message.type = "add" # This was "search" in the previous version. +## Read-write handler - def handle(message, messenger) - $log.info "Droonga::Plugins::SampleLoggerPlugin", :message => message - end - end - end - end -end -~~~ +(TBD) -Restart `fluentd`: +### Directory Structure -~~~ -# kill $(cat fluentd.pid) -# RUBYLIB=./lib fluentd --config fluentd.conf --log fluentd.log --daemon fluentd.pid -~~~ +(TBD) -Let's send a request to Droonga Engine. -Here, we use the first line of `stores.json`. +### Create a plugin -add-store.json: +(TBD) -~~~ -{"dataset":"Starbucks","type":"add","body":{"table":"Store","key":"1st Avenue & 75th St. - New York NY (W)","values":{"location":"40.770262,-73.954798"}}} -~~~ +### Activate the plugin with `catalog.json` -Send it to the engine: +(TBD) -~~~ -# droonga-request --tag starbucks add-store.json -~~~ +### Run -You will see no output for `droonga-request` execution because out `sample-logger` plugin traps the `add` request. +(TBD) -Instead, you will see results like this in `fluentd.log`: +### Test -~~~ -2014-02-17 16:29:18 +0900 [info]: Droonga::Plugins::SampleLoggerPlugin message=#<Droonga::HandlerMessage:0x007f7f6a66c4c0 @raw={"dataset"=>"Starbucks", "type"=>"add", "body"=>{"id"=>"localhost:24224/starbucks.#2", "task"=>{"route"=>"localhost:24224/starbucks.000", "step"=>{"command"=>"add", "dataset"=>"Starbucks", "body"=>{"table"=>"Store", "key"=>"1st Avenue & 75th St. - New York NY (W)", "values"=>{"location"=>"40.770262,-73.954798"}}, "key"=>"1st Avenue & 75th St. - New York NY (W)", "type"=>"scatter", "outputs"=>["errors", "success"], "replica"=>"all", "post"=>true, "routes"=>["localhost:24224/starbucks.000", "localhost:24224/starbucks.001"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#2"], "success"=>["localhost:24224/starbucks.#2"]}}, "n_of_inputs"=>0, "values"=>{}}, "descendants"=>{"errors"=>["localhost:24224/starbucks"], "success"=>["localhost:24224/starbucks"]}}, "replyTo"=>{"type"=>"add.result", "to"=>"127.0.0.1:50480/droonga"}, "id" =>"13926 22158.374441", "date"=>"2014-02-17 16:29:18 +0900", "appliedAdapters"=>["Droonga::Plugins::CRUD::Adapter", "Droonga::Plugins::Error::Adapter"]}, @body={"id"=>"localhost:24224/starbucks.#2", "task"=>{"route"=>"localhost:24224/starbucks.000", "step"=>{"command"=>"add", "dataset"=>"Starbucks", "body"=>{"table"=>"Store", "key"=>"1st Avenue & 75th St. - New York NY (W)", "values"=>{"location"=>"40.770262,-73.954798"}}, "key"=>"1st Avenue & 75th St. - New York NY (W)", "type"=>"scatter", "outputs"=>["errors", "success"], "replica"=>"all", "post"=>true, "routes"=>["localhost:24224/starbucks.000", "localhost:24224/starbucks.001"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#2"], "success"=>["localhost:24224/starbucks.#2"]}}, "n_of_inputs"=>0, "values"=>{}}, "descendants"=>{"errors"=>["localhost:24224/starbucks"], "success"=>["localhost:24224/starbucks"]}}, @task={"route"=>"localhost:24224/starbucks.000", "step"=>{"command"=>"add", "dataset"=>"Starbuck s", "bod y"=>{"table"=>"Store", "key"=>"1st Avenue & 75th St. - New York NY (W)", "values"=>{"location"=>"40.770262,-73.954798"}}, "key"=>"1st Avenue & 75th St. - New York NY (W)", "type"=>"scatter", "outputs"=>["errors", "success"], "replica"=>"all", "post"=>true, "routes"=>["localhost:24224/starbucks.000", "localhost:24224/starbucks.001"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#2"], "success"=>["localhost:24224/starbucks.#2"]}}, "n_of_inputs"=>0, "values"=>{}}, @step={"command"=>"add", "dataset"=>"Starbucks", "body"=>{"table"=>"Store", "key"=>"1st Avenue & 75th St. - New York NY (W)", "values"=>{"location"=>"40.770262,-73.954798"}}, "key"=>"1st Avenue & 75th St. - New York NY (W)", "type"=>"scatter", "outputs"=>["errors", "success"], "replica"=>"all", "post"=>true, "routes"=>["localhost:24224/starbucks.000", "localhost:24224/starbucks.001"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#2"], "success"=>["localhost:2 4224/sta rbucks.#2"]}}> -2014-02-17 16:29:18 +0900 [info]: Droonga::Plugins::SampleLoggerPlugin message=#<Droonga::HandlerMessage:0x007f7f6a65ff40 @raw={"dataset"=>"Starbucks", "type"=>"add", "body"=>{"id"=>"localhost:24224/starbucks.#2", "task"=>{"route"=>"localhost:24224/starbucks.001", "step"=>{"command"=>"add", "dataset"=>"Starbucks", "body"=>{"table"=>"Store", "key"=>"1st Avenue & 75th St. - New York NY (W)", "values"=>{"location"=>"40.770262,-73.954798"}}, "key"=>"1st Avenue & 75th St. - New York NY (W)", "type"=>"scatter", "outputs"=>["errors", "success"], "replica"=>"all", "post"=>true, "routes"=>["localhost:24224/starbucks.000", "localhost:24224/starbucks.001"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#2"], "success"=>["localhost:24224/starbucks.#2"]}}, "n_of_inputs"=>0, "values"=>{}}, "descendants"=>{"errors"=>["localhost:24224/starbucks"], "success"=>["localhost:24224/starbucks"]}}, "replyTo"=>{"type"=>"add.result", "to"=>"127.0.0.1:50480/droonga"}, "id" =>"13926 22158.374441", "date"=>"2014-02-17 16:29:18 +0900", "appliedAdapters"=>["Droonga::Plugins::CRUD::Adapter", "Droonga::Plugins::Error::Adapter"]}, @body={"id"=>"localhost:24224/starbucks.#2", "task"=>{"route"=>"localhost:24224/starbucks.001", "step"=>{"command"=>"add", "dataset"=>"Starbucks", "body"=>{"table"=>"Store", "key"=>"1st Avenue & 75th St. - New York NY (W)", "values"=>{"location"=>"40.770262,-73.954798"}}, "key"=>"1st Avenue & 75th St. - New York NY (W)", "type"=>"scatter", "outputs"=>["errors", "success"], "replica"=>"all", "post"=>true, "routes"=>["localhost:24224/starbucks.000", "localhost:24224/starbucks.001"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#2"], "success"=>["localhost:24224/starbucks.#2"]}}, "n_of_inputs"=>0, "values"=>{}}, "descendants"=>{"errors"=>["localhost:24224/starbucks"], "success"=>["localhost:24224/starbucks"]}}, @task={"route"=>"localhost:24224/starbucks.001", "step"=>{"command"=>"add", "dataset"=>"Starbuck s", "bod y"=>{"table"=>"Store", "key"=>"1st Avenue & 75th St. - New York NY (W)", "values"=>{"location"=>"40.770262,-73.954798"}}, "key"=>"1st Avenue & 75th St. - New York NY (W)", "type"=>"scatter", "outputs"=>["errors", "success"], "replica"=>"all", "post"=>true, "routes"=>["localhost:24224/starbucks.000", "localhost:24224/starbucks.001"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#2"], "success"=>["localhost:24224/starbucks.#2"]}}, "n_of_inputs"=>0, "values"=>{}}, @step={"command"=>"add", "dataset"=>"Starbucks", "body"=>{"table"=>"Store", "key"=>"1st Avenue & 75th St. - New York NY (W)", "values"=>{"location"=>"40.770262,-73.954798"}}, "key"=>"1st Avenue & 75th St. - New York NY (W)", "type"=>"scatter", "outputs"=>["errors", "success"], "replica"=>"all", "post"=>true, "routes"=>["localhost:24224/starbucks.000", "localhost:24224/starbucks.001"], "n_of_expects"=>0, "descendants"=>{"errors"=>["localhost:24224/starbucks.#2"], "success"=>["localhost:2 4224/sta rbucks.#2"]}}> -~~~ +(TBD) -In `add` case, two log lines are shown for one request. This is because we have configured to have two replicas for each partition. +### Design input and output -In order to be consistent, `add` command must reach all of the replicas of the partition, but not the other partitions. -As a consequence, `localhost:24224/starbucks.000` and `localhost:24224/starbucks.001` are chosen. +(TBD) ## Conclusion -------------- next part -------------- HTML����������������������������...Download