[groonga-dev,01708] Re: 仮想メモリサイズを超えるmroongaのインデックス構築について

Back to archive index

Naoya Murakami visio****@gmail*****
2013年 8月 28日 (水) 08:47:20 JST


お世話になっております。村上です。

再現させてみました!
2個前のgrn_ii_buffer_commit: starというと、1つ前のインデックスになってしまい、かなり長かったので、
途中は省略しました。省略の間は、grn_ii〜は出力されていません。

予想とは少し違うかもしれませんが何かわかりそうですか?

ブランチのソースから、too many postings回避のため、以下を書き換えています。
3分割したときは、これを書き換えても、うまくいっていたので、これが原因かは
わかりませんが、ii.cなので、ひょっとしたら、これが悪いのかもしれません。
そうだったら、ごめんなさい。

#define GRN_II_MAX_TF 0x1ffff → #define GRN_II_MAX_TF 0x1fffff

2013-08-27 21:25:49.011134|n|d4172700|grn_ii_buffer_commit: start
2013-08-27 21:25:49.011187|n|d4172700|grn_ii_buffer_commit: block_pos:
13500648
2013-08-27 21:25:49.116706|n|d4172700|flushed: 0 encsize:20250985
2013-08-27 21:25:49.116752|n|d4172700|grn_ii_buffer_commit: tmpfd: 222
2013-08-27 21:25:49.116760|n|d4172700|grn_ii_buffer_commit: close tmpfd
2013-08-27 21:25:49.116772|n|d4172700|grn_ii_buffer_commit: block_buf:
0x7f0fe73fc010
2013-08-27 21:25:49.116780|n|d4172700|grn_ii_buffer_commit: free block_buf
2013-08-27 21:25:49.121239|n|d4172700|grn_ii_buffer_commit: counters: (nil)
2013-08-27 21:25:49.121257|n|d4172700|grn_ii_buffer_commit: free counters
2013-08-27 21:25:49.121274|n|d4172700|grn_ii_buffer_commit:
update_buffer_size: 10
2013-08-27 21:25:49.121285|n|d4172700|nblocks=1, update_buffer_size=13500648
2013-08-27 21:25:49.121292|n|d4172700|grn_ii_buffer_commit: datavec_init
2013-08-27 21:25:49.121299|n|d4172700|grn_ii_buffer_commit: datavec_init: 0
2013-08-27 21:25:49.121315|n|d4172700|grn_ii_buffer_commit: opened tmpfd:
222
2013-08-27 21:25:49.121323|n|d4172700|grn_ii_buffer_commit: fetch: 1
2013-08-27 21:25:49.133549|n|d4172700|grn_ii_buffer_commit: hits: 1
2013-08-27 21:25:49.381154|n|d4172700|nterms=1 chunk=1002008 total=978KB
2013-08-27 21:25:49.381208|n|d4172700|grn_ii_buffer_commit: databec_fin
2013-08-27 21:25:49.388169|n|d4172700|tmpfile_size:20250985 >
total_chunk_size:1002008
2013-08-27 21:25:49.395042|n|d4172700|DDL:table_create ftext-ftext
2013-08-27 21:25:49.397128|n|d4172700|DDL:column_create ftext-ftext index
2013-08-27 21:25:49.405061|n|d4172700|DDL:set_source ftext-ftext.index
ftext.title,ftext.abstract,ftext.claims,ftext.description_19xx,ftext.description_20xx,ftext.property
2013-08-27 21:25:55.149552|n|d4172700|flushing:0 npostings:16760470
2013-08-27 21:25:55.720210|n|d4172700|flushed: 0 encsize:31696156
2013-08-27 21:26:01.306507|n|d4172700|flushing:1 npostings:16766538
2013-08-27 21:26:01.742487|n|d4172700|flushed: 1 encsize:31594296
・・・
2013-08-28 05:38:51.142670|n|d4172700|flushing:5311 npostings:16734447
2013-08-28 05:38:51.591159|n|d4172700|flushed: 5311 encsize:31361918
2013-08-28 05:38:56.401537|n|d4172700|grn_ii_buffer_commit: start
2013-08-28 05:38:56.401605|n|d4172700|grn_ii_buffer_commit: block_pos:
14497232
2013-08-28 05:38:56.401615|n|d4172700|grn_ii_buffer_commit: flush
2013-08-28 05:38:56.401623|n|d4172700|flushing:5312 npostings:14497232
2013-08-28 05:38:56.964763|n|d4172700|flushed: 5312 encsize:27181898
2013-08-28 05:38:56.964809|n|d4172700|grn_ii_buffer_commit: tmpfd: 225
2013-08-28 05:38:56.964818|n|d4172700|grn_ii_buffer_commit: close tmpfd
2013-08-28 05:38:56.964831|n|d4172700|grn_ii_buffer_commit: block_buf:
0x7f0fe73fc010
2013-08-28 05:38:56.964838|n|d4172700|grn_ii_buffer_commit: free block_buf
2013-08-28 05:38:56.969587|n|d4172700|grn_ii_buffer_commit: counters: (nil)
2013-08-28 05:38:56.969606|n|d4172700|grn_ii_buffer_commit: free counters
2013-08-28 05:38:56.969989|n|d4172700|grn_ii_buffer_commit:
update_buffer_size: 10
2013-08-28 05:38:56.970005|n|d4172700|nblocks=5313,
update_buffer_size=95856484978
2013-08-28 05:38:56.970014|n|d4172700|grn_ii_buffer_commit: datavec_init
2013-08-28 05:38:56.970023|n|d4172700|grn_ii_buffer_commit: datavec_init: 0
2013-08-28 05:38:56.970042|n|d4172700|grn_ii_buffer_commit: opened tmpfd:
225
2013-08-28 05:38:56.970050|n|d4172700|grn_ii_buffer_commit: fetch: 5313
2013-08-28 05:39:23.951606|n|d4172700|grn_ii_buffer_commit: hits: 5313
2013-08-28 05:39:32.901323|n|d4172700|nterms=6 chunk=1230 total=1KB
2013-08-28 05:39:40.755374|n|d4172700|nterms=1 chunk=1003688391
total=980165KB
2013-08-28 05:39:40.768927|n|d4172700|nterms=2 chunk=1018907 total=981160KB
2013-08-28 05:39:43.871382|n|d4172700|nterms=9 chunk=164 total=981160KB
2013-08-28 05:39:51.514877|n|d4172700|nterms=12 chunk=899969567
total=1860037KB
・・・
2013-08-28 05:40:48.445918|n|d4172700|nterms=163 chunk=736791
total=5903876KB
2013-08-28 05:40:48.495673|n|d4172700|nterms=384 chunk=726545
total=5904586KB
2013-08-28 05:40:48.531348|n|d4172700|nterms=634 chunk=503434
total=5905077KB
2013-08-28 05:40:48.560509|n|d4172700|nterms=188 chunk=506237
total=5905572KB
2013-08-28 05:40:48.599969|n|d4172700|nterms=298 chunk=959292
total=5906508KB
2013-08-28 05:40:48.660974|n|d4172700|nterms=236 chunk=2186602
total=5908644KB
2013-08-28 05:40:53.354674|n|5b7af720|mroonga 3.07 started.


20:40:48 UTC - mysqld got signal 11 ;
Thread pointer: 0x2833300
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7f11d4171e28 thread_stack 0x40000
/usr/local/mysql/bin/mysqld(my_print_stacktrace+0x35)[0x8d77e5]
/usr/local/mysql/bin/mysqld(handle_fatal_signal+0x40b)[0x64cdfb]
/lib64/libpthread.so.0[0x318c00f500]
/lib64/libc.so.6[0x318bc78962]
/usr/lib64/libgroonga.so.0(grn_free_default+0x31)[0x7f11d61478f1]
/usr/lib64/libgroonga.so.0(+0x10b0d4)[0x7f11d62310d4]
/usr/lib64/libgroonga.so.0(+0x128e27)[0x7f11d624ee27]
/usr/lib64/libgroonga.so.0(grn_ii_buffer_commit+0x600)[0x7f11d6250e10]
/usr/lib64/libgroonga.so.0(grn_ii_build+0x4b5)[0x7f11d62516d5]
/usr/lib64/libgroonga.so.0(grn_obj_set_info+0xaec)[0x7f11d617c18c]
/usr/local/mysql/lib/plugin/ha_mroonga.so(_ZN10ha_mroonga20storage_create_indexEP5TABLEPKcP8_grn_objP16st_mroonga_shareP6st_keyPS5_SA_j+0x369)[0x7f10aa1a2f29]
/usr/local/mysql/lib/plugin/ha_mroonga.so(_ZN10ha_mroonga22storage_enable_indexesEj+0x274)[0x7f10aa1adc34]
/usr/local/mysql/bin/mysqld(_Z17mysql_alter_tableP3THDPcS1_P24st_ha_create_informationP10TABLE_LISTP10Alter_infojP8st_orderb+0x2154)[0x7232c4]
/usr/local/mysql/bin/mysqld(_ZN19Sql_cmd_alter_table7executeEP3THD+0x43c)[0x8161ec]
/usr/local/mysql/bin/mysqld(_Z21mysql_execute_commandP3THD+0x1af1)[0x6d1331]
/usr/local/mysql/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x4c7)[0x6d4637]
/usr/local/mysql/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1164)[0x6d5894]
/usr/local/mysql/bin/mysqld(_Z24do_handle_one_connectionP3THD+0x10d)[0x69e9cd]
/usr/local/mysql/bin/mysqld(handle_one_connection+0x42)[0x69eb02]
/usr/local/mysql/bin/mysqld(pfs_spawn_thread+0x127)[0x91d527]
/lib64/libpthread.so.0[0x318c007851]
/lib64/libc.so.6(clone+0x6d)[0x318bce890d]

以上、よろしくお願いします。



groonga-dev メーリングリストの案内
Back to archive index