<!DOCTYPE html>
<html>
<head>
</head>
<body>
<dl style="line-height: 1.5; margin-left: 2em">
<dt style="clear: both; float: left; font-weight: bold; width: 8em">Author</dt>
<dd style="margin-left: 8.5em">Kouhei Sutou <kou****@clear*****></dd>
<dt style="clear: both; float: left; font-weight: bold; width: 8em">Date</dt>
<dd style="margin-left: 8.5em">2018-09-20 17:15:55 +0900 (Thu, 20 Sep 2018)</dd>
<dt style="clear: both; float: left; font-weight: bold; width: 8em">Revision</dt>
<dd style="margin-left: 8.5em"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1">95f8fd3600bc16c640808bb30c9dd49cae23ded1</a></dd>
<dt style="clear: both; float: left; font-weight: bold; width: 8em">Message</dt>
<dd style="margin-left: 8.5em"><pre style="border: 1px solid #aaa; font-family: Consolas, Menlo, "Liberation Mono", Courier, monospace; line-height: 1.2; padding: 0.5em; width: auto">TokenNgram: add unify_digit option
TokenNgram("unify_digit", false) == TokenBigramSplitDigit (not exist ;p)</pre></dd>
<dt style="clear: both; float: left; font-weight: bold; width: 8em">Added files</dt>
<dd style="margin-left: 8.5em">
<ul>
<li><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366">test/command/suite/tokenizers/ngram/unify_digit.expected</a></li>
<li><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-537bb19e93f3d4eb2c7baafe2e0eb733">test/command/suite/tokenizers/ngram/unify_digit.test</a></li>
</ul>
</dd>
<dt style="clear: both; float: left; font-weight: bold; width: 8em">Modified files</dt>
<dd style="margin-left: 8.5em">
<ul>
<li><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8e">lib/tokenizers.c</a></li>
</ul>
</dd>
</dl>
<div class="diff-section" style="clear: both">
<table style="border-collapse: collapse; border: 1px solid #aaa">
<thead>
<tr class="diff-header" style="border: 1px solid #aaa">
<td colspan="3">
<pre style="border: 0; font-family: Consolas, Menlo, "Liberation Mono", Courier, monospace; line-height: 1.2; margin: 0; padding: 0.5em; white-space: normal; width: auto"><span class="diff-header" style="background-color: #eaf2f5; color: #999999; display: block; white-space: pre"> Modified: lib/tokenizers.c (+6 -0)</span>
<span class="diff-header-mark" style="background-color: #eaf2f5; color: #999999; display: block; white-space: pre">===================================================================</span>
</pre>
</td>
</tr>
</thead>
<tbody>
<tr>
<th class="diff-line-number" style="border: 1px solid #aaa">
<pre style="border: 0; font-family: Consolas, Menlo, "Liberation Mono", Courier, monospace; line-height: 1.2; margin: 0; padding: 0.5em; white-space: normal; width: auto"><span class="diff-line-number-hunk-header" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eL735">...</a></span>
<span class="diff-line-number-not-changed" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eL736">736</a></span>
<span class="diff-line-number-not-changed" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eL737">737</a></span>
<span class="diff-line-number-not-changed" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eL738">738</a></span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-not-changed" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eL739">739</a></span>
<span class="diff-line-number-not-changed" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eL740">740</a></span>
<span class="diff-line-number-not-changed" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eL741">741</a></span>
</pre>
</th>
<th class="diff-line-number" style="border: 1px solid #aaa">
<pre style="border: 0; font-family: Consolas, Menlo, "Liberation Mono", Courier, monospace; line-height: 1.2; margin: 0; padding: 0.5em; white-space: normal; width: auto"><span class="diff-line-number-hunk-header" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eR735">...</a></span>
<span class="diff-line-number-not-changed" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eR736">736</a></span>
<span class="diff-line-number-not-changed" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eR737">737</a></span>
<span class="diff-line-number-not-changed" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eR738">738</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eR739">739</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eR740">740</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eR741">741</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eR742">742</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eR743">743</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eR744">744</a></span>
<span class="diff-line-number-not-changed" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eR745">745</a></span>
<span class="diff-line-number-not-changed" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eR746">746</a></span>
<span class="diff-line-number-not-changed" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-bb0ba8ce6a2347aa780a747cf6452b8eR747">747</a></span>
</pre>
</th>
<td class="diff-content" style="border: 1px solid #aaa">
<pre style="border: 0; font-family: Consolas, Menlo, "Liberation Mono", Courier, monospace; line-height: 1.2; margin: 0; padding: 0.5em; white-space: normal; width: auto"><span class="diff-hunk-header" style="background-color: #eaf2f5; color: #999999; display: block; white-space: pre">@@ -736,6 +736,12 @@ <span class="diff-context" style="background-color: #ffffaa; color: #000000">ngram_open_options(grn_ctx *ctx,</span></span>
<span class="diff-not-changed" style="display: block; white-space: pre"> raw_options,</span>
<span class="diff-not-changed" style="display: block; white-space: pre"> i,</span>
<span class="diff-not-changed" style="display: block; white-space: pre"> options->unify_alphabet);</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ } else if (GRN_RAW_STRING_EQUAL_CSTRING(name_raw, "unify_digit")) {</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ options->unify_digit =</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ grn_vector_get_element_bool(ctx,</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ raw_options,</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ i,</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ options->unify_digit);</span>
<span class="diff-not-changed" style="display: block; white-space: pre"> } else if (GRN_RAW_STRING_EQUAL_CSTRING(name_raw, "unify_symbol")) {</span>
<span class="diff-not-changed" style="display: block; white-space: pre"> options->unify_symbol =</span>
<span class="diff-not-changed" style="display: block; white-space: pre"> grn_vector_get_element_bool(ctx,</span>
</pre>
</td>
</tr>
</tbody>
</table>
<table style="border-collapse: collapse; border: 1px solid #aaa">
<thead>
<tr class="diff-header" style="border: 1px solid #aaa">
<td colspan="3">
<pre style="border: 0; font-family: Consolas, Menlo, "Liberation Mono", Courier, monospace; line-height: 1.2; margin: 0; padding: 0.5em; white-space: normal; width: auto"><span class="diff-header" style="background-color: #eaf2f5; color: #999999; display: block; white-space: pre"> Added: test/command/suite/tokenizers/ngram/unify_digit.expected (+40 -0) 100644</span>
<span class="diff-header-mark" style="background-color: #eaf2f5; color: #999999; display: block; white-space: pre">===================================================================</span>
</pre>
</td>
</tr>
</thead>
<tbody>
<tr>
<th class="diff-line-number" style="border: 1px solid #aaa">
<pre style="border: 0; font-family: Consolas, Menlo, "Liberation Mono", Courier, monospace; line-height: 1.2; margin: 0; padding: 0.5em; white-space: normal; width: auto"><span class="diff-line-number-hunk-header" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366">...</a></span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
</pre>
</th>
<th class="diff-line-number" style="border: 1px solid #aaa">
<pre style="border: 0; font-family: Consolas, Menlo, "Liberation Mono", Courier, monospace; line-height: 1.2; margin: 0; padding: 0.5em; white-space: normal; width: auto"><span class="diff-line-number-hunk-header" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366">...</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R1">1</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R2">2</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R3">3</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R4">4</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R5">5</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R6">6</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R7">7</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R8">8</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R9">9</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R10">10</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R11">11</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R12">12</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R13">13</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R14">14</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R15">15</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R16">16</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R17">17</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R18">18</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R19">19</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R20">20</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R21">21</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R22">22</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R23">23</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R24">24</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R25">25</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R26">26</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R27">27</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R28">28</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R29">29</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R30">30</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R31">31</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R32">32</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R33">33</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R34">34</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R35">35</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R36">36</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R37">37</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R38">38</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R39">39</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-06cad7655cc50c0d774e5b4bd5860366R40">40</a></span>
</pre>
</th>
<td class="diff-content" style="border: 1px solid #aaa">
<pre style="border: 0; font-family: Consolas, Menlo, "Liberation Mono", Courier, monospace; line-height: 1.2; margin: 0; padding: 0.5em; white-space: normal; width: auto"><span class="diff-hunk-header" style="background-color: #eaf2f5; color: #999999; display: block; white-space: pre">@@ -0,0 +1,40 @@</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+tokenize 'TokenNgram("unify_digit", false)' "012345" NormalizerAuto</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+[</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ [</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ 0,</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ 0.0,</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ 0.0</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ ],</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ [</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ {</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "value": "01",</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "position": 0,</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "force_prefix": false</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ },</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ {</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "value": "12",</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "position": 1,</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "force_prefix": false</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ },</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ {</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "value": "23",</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "position": 2,</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "force_prefix": false</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ },</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ {</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "value": "34",</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "position": 3,</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "force_prefix": false</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ },</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ {</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "value": "45",</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "position": 4,</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "force_prefix": false</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ },</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ {</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "value": "5",</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "position": 5,</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "force_prefix": false</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ }</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ ]</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+]</span>
</pre>
</td>
</tr>
</tbody>
</table>
<table style="border-collapse: collapse; border: 1px solid #aaa">
<thead>
<tr class="diff-header" style="border: 1px solid #aaa">
<td colspan="3">
<pre style="border: 0; font-family: Consolas, Menlo, "Liberation Mono", Courier, monospace; line-height: 1.2; margin: 0; padding: 0.5em; white-space: normal; width: auto"><span class="diff-header" style="background-color: #eaf2f5; color: #999999; display: block; white-space: pre"> Added: test/command/suite/tokenizers/ngram/unify_digit.test (+4 -0) 100644</span>
<span class="diff-header-mark" style="background-color: #eaf2f5; color: #999999; display: block; white-space: pre">===================================================================</span>
</pre>
</td>
</tr>
</thead>
<tbody>
<tr>
<th class="diff-line-number" style="border: 1px solid #aaa">
<pre style="border: 0; font-family: Consolas, Menlo, "Liberation Mono", Courier, monospace; line-height: 1.2; margin: 0; padding: 0.5em; white-space: normal; width: auto"><span class="diff-line-number-hunk-header" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-537bb19e93f3d4eb2c7baafe2e0eb733">...</a></span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
<span class="diff-line-number-nothing" style="display: block; white-space: pre"> </span>
</pre>
</th>
<th class="diff-line-number" style="border: 1px solid #aaa">
<pre style="border: 0; font-family: Consolas, Menlo, "Liberation Mono", Courier, monospace; line-height: 1.2; margin: 0; padding: 0.5em; white-space: normal; width: auto"><span class="diff-line-number-hunk-header" style="display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-537bb19e93f3d4eb2c7baafe2e0eb733">...</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-537bb19e93f3d4eb2c7baafe2e0eb733R1">1</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-537bb19e93f3d4eb2c7baafe2e0eb733R2">2</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-537bb19e93f3d4eb2c7baafe2e0eb733R3">3</a></span>
<span class="diff-line-number-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre"><a href="https://github.com/groonga/groonga/commit/95f8fd3600bc16c640808bb30c9dd49cae23ded1#diff-537bb19e93f3d4eb2c7baafe2e0eb733R4">4</a></span>
</pre>
</th>
<td class="diff-content" style="border: 1px solid #aaa">
<pre style="border: 0; font-family: Consolas, Menlo, "Liberation Mono", Courier, monospace; line-height: 1.2; margin: 0; padding: 0.5em; white-space: normal; width: auto"><span class="diff-hunk-header" style="background-color: #eaf2f5; color: #999999; display: block; white-space: pre">@@ -0,0 +1,4 @@</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+tokenize \</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ 'TokenNgram("unify_digit", false)' \</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ "012345" \</span>
<span class="diff-added" style="background-color: #eaffea; color: #000000; display: block; white-space: pre">+ NormalizerAuto</span>
</pre>
</td>
</tr>
</tbody>
</table>
</div>
</body>
</html>