Splunk SPL Examplesについて

Splunk SPL Examples
の日本語版を作ろうという話があるので、まずは既存のAPPがどうなっているのかを調べてみる。

https://github.com/takashikomatsubara/dev_app_splreference
こちらがこれから作るやつ

https://app.slack.com/client/T4P3ZML6T/C01978DQE7J/details/info
GOJASのSlack Channelはこちら

tree

.
├── README.txt
├── appserver
│   └── static
│       ├── dbinspect-example1.png
│       ├── eval-image1.png
│       ├── geom-image1.png
│       ├── join-image1.png
│       ├── metadata-image1.png
│       └── stats-image1.png
├── bin
│   └── README
├── default
│   ├── app.conf
│   ├── data
│   │   ├── models
│   │   │   └── splexamples_tutorial.json
│   │   └── ui
│   │       ├── nav
│   │       │   └── default.xml
│   │       ├── panels
│   │       │   ├── abstract_desc.xml
│   │       │   ├── accum_desc.xml
│   │       │   ├── addcoltotals_desc.xml
│   │       │   ├── addinfo_desc.xml
│   │       │   ├── addtotals_desc.xml
│   │       │   ├── analyzefields_desc.xml
│   │       │   ├── anomalies_desc.xml
│   │       │   ├── anomalousvalue_desc.xml
│   │       │   ├── anomalydetection_desc.xml
│   │       │   ├── append_desc.xml
│   │       │   ├── appendcols_desc.xml
│   │       │   ├── appendpipe_desc.xml
│   │       │   ├── arules_desc.xml
│   │       │   ├── associate_desc.xml
│   │       │   ├── audit_desc.xml
│   │       │   ├── autoregress_desc.xml
│   │       │   ├── bin_desc.xml
│   │       │   ├── bucketdir_desc.xml
│   │       │   ├── chart_desc.xml
│   │       │   ├── cluster_desc.xml
│   │       │   ├── cofilter_desc.xml
│   │       │   ├── collect_desc.xml
│   │       │   ├── concurrency_desc.xml
│   │       │   ├── contingency_desc.xml
│   │       │   ├── convert_desc.xml
│   │       │   ├── correlate_desc.xml
│   │       │   ├── crawl_desc.xml
│   │       │   ├── datamodel_desc.xml
│   │       │   ├── date_and_time_format_variables.xml
│   │       │   ├── dbinspect_desc.xml
│   │       │   ├── dedup_desc.xml
│   │       │   ├── delete_desc.xml
│   │       │   ├── delta_desc.xml
│   │       │   ├── diff_desc.xml
│   │       │   ├── erex_desc.xml
│   │       │   ├── eval_desc.xml
│   │       │   ├── eval_functions_desc.xml
│   │       │   ├── eventcount_desc.xml
│   │       │   ├── eventstats_desc.xml
│   │       │   ├── extract_desc.xml
│   │       │   ├── fieldformat_desc.xml
│   │       │   ├── fields_desc.xml
│   │       │   ├── fieldsummary_desc.xml
│   │       │   ├── filldown_desc.xml
│   │       │   ├── fillnull_desc.xml
│   │       │   ├── findtypes_desc.xml
│   │       │   ├── folderize_desc.xml
│   │       │   ├── foreach_desc.xml
│   │       │   ├── format_desc.xml
│   │       │   ├── from_desc.xml
│   │       │   ├── gauge_desc.xml
│   │       │   ├── gentimes_desc.xml
│   │       │   ├── geom_desc.xml
│   │       │   ├── geomfilter_desc.xml
│   │       │   ├── geostats_desc.xml
│   │       │   ├── head_desc.xml
│   │       │   ├── highlight_desc.xml
│   │       │   ├── history_desc.xml
│   │       │   ├── iconify_desc.xml
│   │       │   ├── input_desc.xml
│   │       │   ├── inputcsv_desc.xml
│   │       │   ├── inputlookup_desc.xml
│   │       │   ├── iplocation_desc.xml
│   │       │   ├── join_desc.xml
│   │       │   ├── kmeans_desc.xml
│   │       │   ├── kvform_desc.xml
│   │       │   ├── loadjob_desc.xml
│   │       │   ├── localize_desc.xml
│   │       │   ├── localop_desc.xml
│   │       │   ├── lookup_desc.xml
│   │       │   ├── makecontinuous_desc.xml
│   │       │   ├── makemv_desc.xml
│   │       │   ├── makeresults_desc.xml
│   │       │   ├── map_desc.xml
│   │       │   ├── metadata_desc.xml
│   │       │   ├── metasearch_desc.xml
│   │       │   ├── multikv_desc.xml
│   │       │   ├── multisearch_desc.xml
│   │       │   ├── mvcombine_desc.xml
│   │       │   ├── mvexpand_desc.xml
│   │       │   ├── nomv_desc.xml
│   │       │   ├── outlier_desc.xml
│   │       │   ├── outputcsv_desc.xml
│   │       │   ├── outputlookup_desc.xml
│   │       │   ├── outputtext_desc.xml
│   │       │   ├── overlap_desc.xml
│   │       │   ├── pivot_desc.xml
│   │       │   ├── predict_desc.xml
│   │       │   ├── rangemap_desc.xml
│   │       │   ├── rare_desc.xml
│   │       │   ├── regex_desc.xml
│   │       │   ├── relevancy_desc.xml
│   │       │   ├── reltime_desc.xml
│   │       │   ├── rename_desc.xml
│   │       │   ├── replace_desc.xml
│   │       │   ├── rest_desc.xml
│   │       │   ├── return_desc.xml
│   │       │   ├── reverse_desc.xml
│   │       │   ├── rex_desc.xml
│   │       │   ├── rtorder_desc.xml
│   │       │   ├── run_desc.xml
│   │       │   ├── savedsearch_desc.xml
│   │       │   ├── script_desc.xml
│   │       │   ├── scrub_desc.xml
│   │       │   ├── search_cmd_desc.xml
│   │       │   ├── searchtxn_desc.xml
│   │       │   ├── selfjoin_desc.xml
│   │       │   ├── sendemail_desc.xml
│   │       │   ├── set_desc.xml
│   │       │   ├── setfields_desc.xml
│   │       │   ├── sichart_desc.xml
│   │       │   ├── sirare_desc.xml
│   │       │   ├── sistats_desc.xml
│   │       │   ├── sitimechart_desc.xml
│   │       │   ├── sitop_desc.xml
│   │       │   ├── sort_desc.xml
│   │       │   ├── spath_desc.xml
│   │       │   ├── statistical_charting_functions_desc.xml
│   │       │   ├── stats_desc.xml
│   │       │   ├── strcat_desc.xml
│   │       │   ├── streamstats_desc.xml
│   │       │   ├── table_desc.xml
│   │       │   ├── tags_desc.xml
│   │       │   ├── tail_desc.xml
│   │       │   ├── time_modifiers.xml
│   │       │   ├── timechart_desc.xml
│   │       │   ├── timewrap_desc.xml
│   │       │   ├── top_desc.xml
│   │       │   ├── transaction_desc.xml
│   │       │   ├── transpose_desc.xml
│   │       │   ├── trendline_desc.xml
│   │       │   ├── tscollect_desc.xml
│   │       │   ├── tstats_desc.xml
│   │       │   ├── typeahead_desc.xml
│   │       │   ├── typelearner_desc.xml
│   │       │   ├── typer_desc.xml
│   │       │   ├── uniq_desc.xml
│   │       │   ├── untable_desc.xml
│   │       │   ├── where_desc.xml
│   │       │   ├── x11_desc.xml
│   │       │   ├── xmlkv_desc.xml
│   │       │   ├── xmlunescape_desc.xml
│   │       │   ├── xpath_desc.xml
│   │       │   └── xyseries_desc.xml
│   │       └── views
│   │           ├── abstract.xml
│   │           ├── accum.xml
│   │           ├── addcoltotals.xml
│   │           ├── addinfo.xml
│   │           ├── addtotals.xml
│   │           ├── analysefields.xml
│   │           ├── anomalies.xml
│   │           ├── anomalousvalue.xml
│   │           ├── anomalydetection.xml
│   │           ├── append.xml
│   │           ├── appendcols.xml
│   │           ├── appendpipe.xml
│   │           ├── arules.xml
│   │           ├── associate.xml
│   │           ├── audit.xml
│   │           ├── autoregress.xml
│   │           ├── bin.xml
│   │           ├── bucketdir.xml
│   │           ├── chart.xml
│   │           ├── cluster.xml
│   │           ├── cofilter.xml
│   │           ├── collect.xml
│   │           ├── concurrency.xml
│   │           ├── contingency.xml
│   │           ├── convert.xml
│   │           ├── correlate.xml
│   │           ├── crawl.xml
│   │           ├── data_and_time_format_variables.xml
│   │           ├── datamodel.xml
│   │           ├── dbinspect.xml
│   │           ├── dedup.xml
│   │           ├── delete.xml
│   │           ├── delta.xml
│   │           ├── diff.xml
│   │           ├── erex.xml
│   │           ├── eval.xml
│   │           ├── evaluation_functions.xml
│   │           ├── eventcount.xml
│   │           ├── eventstats.xml
│   │           ├── extract.xml
│   │           ├── fieldformat.xml
│   │           ├── fields.xml
│   │           ├── fieldsummary.xml
│   │           ├── filldown.xml
│   │           ├── fillnull.xml
│   │           ├── find_search_command.xml
│   │           ├── findtypes.xml
│   │           ├── folderize.xml
│   │           ├── foreach.xml
│   │           ├── format.xml
│   │           ├── from.xml
│   │           ├── gauge.xml
│   │           ├── gentimes.xml
│   │           ├── geom.xml
│   │           ├── geomfilter.xml
│   │           ├── geostats.xml
│   │           ├── head.xml
│   │           ├── highlight.xml
│   │           ├── history.xml
│   │           ├── iconify.xml
│   │           ├── input.xml
│   │           ├── inputcsv.xml
│   │           ├── inputlookup.xml
│   │           ├── introduction.xml
│   │           ├── iplocation.xml
│   │           ├── join.xml
│   │           ├── kmeans.xml
│   │           ├── kvform.xml
│   │           ├── loadjob.xml
│   │           ├── localize.xml
│   │           ├── localop.xml
│   │           ├── lookup.xml
│   │           ├── makecontinuous.xml
│   │           ├── makemv.xml
│   │           ├── makeresults.xml
│   │           ├── map.xml
│   │           ├── metadata.xml
│   │           ├── metasearch.xml
│   │           ├── multikv.xml
│   │           ├── multisearch.xml
│   │           ├── mvcombine.xml
│   │           ├── mvexpand.xml
│   │           ├── nomv.xml
│   │           ├── outlier.xml
│   │           ├── outputcsv.xml
│   │           ├── outputlookup.xml
│   │           ├── outputtext.xml
│   │           ├── overlap.xml
│   │           ├── pivot.xml
│   │           ├── predict.xml
│   │           ├── rangemap.xml
│   │           ├── rare.xml
│   │           ├── regex.xml
│   │           ├── relevancy.xml
│   │           ├── reltime.xml
│   │           ├── rename.xml
│   │           ├── replace.xml
│   │           ├── rest.xml
│   │           ├── return.xml
│   │           ├── reverse.xml
│   │           ├── rex.xml
│   │           ├── rtorder.xml
│   │           ├── run.xml
│   │           ├── savedsearch.xml
│   │           ├── script.xml
│   │           ├── scrub.xml
│   │           ├── search_cmd.xml
│   │           ├── searchtxn.xml
│   │           ├── selfjoin.xml
│   │           ├── sendemail.xml
│   │           ├── set.xml
│   │           ├── setfields.xml
│   │           ├── sichart.xml
│   │           ├── sirare.xml
│   │           ├── sistats.xml
│   │           ├── sitimechart.xml
│   │           ├── sitop.xml
│   │           ├── sort.xml
│   │           ├── spath.xml
│   │           ├── statistical_and_charting_functions.xml
│   │           ├── stats.xml
│   │           ├── strcat.xml
│   │           ├── streamstats.xml
│   │           ├── table.xml
│   │           ├── tags.xml
│   │           ├── tail.xml
│   │           ├── time_modifiers.xml
│   │           ├── timechart.xml
│   │           ├── timewrap.xml
│   │           ├── top.xml
│   │           ├── transaction.xml
│   │           ├── transpose.xml
│   │           ├── trendline.xml
│   │           ├── tscollect.xml
│   │           ├── tstats.xml
│   │           ├── typeahead.xml
│   │           ├── typelearner.xml
│   │           ├── typer.xml
│   │           ├── uniq.xml
│   │           ├── untable.xml
│   │           ├── where.xml
│   │           ├── x11.xml
│   │           ├── xmlkv.xml
│   │           ├── xmlunescape.xml
│   │           ├── xpath.xml
│   │           └── xyseries.xml
│   ├── datamodels.conf
│   ├── eventgen.conf
│   ├── eventtypes.conf
│   ├── indexes.conf
│   ├── props.conf
│   ├── savedsearches.conf
│   └── transforms.conf
├── eventgen_replace
│   └── webhosts.csv
├── lookups
│   ├── cmdlookup.csv
│   ├── expected_hosts.csv
│   ├── httpstatuscodes.csv
│   ├── prices.csv
│   ├── usertoip.csv
│   └── vendors.csv
├── metadata
│   ├── default.meta
│   └── local.meta
├── samples
│   ├── earthquake.log
│   ├── email.log
│   ├── linux.vmstat
│   ├── mailsv.secure.log
│   ├── noise.cpu
│   ├── noise.df
│   ├── noise.iostat
│   ├── vendor_sales.log
│   ├── www1.access.log
│   ├── www1.secure.log
│   ├── www2.access.log
│   ├── www2.secure.log
│   ├── www3.access.log
│   └── www3.secure.log
└── static
    ├── appIcon.png
    ├── appIconAlt.png
    ├── appIconAlt_2x.png
    ├── appIcon_2x.png
    └── appLogo.png

cmd_panels.spl

| inputlookup cmdlookup

全部で142のSPLコマンドがある。
作った人すごいな。

top_desc.xml

op_desc.xml

<panel>
  <html>
    <h2>
      <span id="Description">Description</span>
    </h2>
    <p>Displays the most common values of a field.</p>
    <p>Finds the most frequent tuple of values of all fields in the field list, along with a count and percentage. If the optional by-clause is included, the command finds the most frequent values for each distinct tuple of values of the group-by fields.</p>
    <h2>
      <span id="Syntax">Syntax</span>
    </h2>
    <p>top [&lt;N&gt;] [&lt;top-options&gt;...] &lt;field-list&gt; [&lt;by-clause&gt;]</p>
    <h3>
      <span id="Required_arguments">Required arguments</span>
    </h3>
    <dl>
      <dt>&lt;field-list&gt;</dt>
      <dd>
        <strong>Syntax:</strong>&lt;field&gt;, &lt;field&gt;, ...</dd>
      <dd>
        <strong>Description:</strong>Comma-delimited list of field names.</dd>
    </dl>
    <h3>
      <span id="Optional_arguments">Optional arguments</span>
    </h3>
    <dl>
      <dt>&lt;N&gt;</dt>
      <dd>
        <strong>Syntax:</strong>&lt;int&gt;</dd>
      <dd>
        <strong>Description:</strong>The number of results to return.</dd>
    </dl>
    <dl>
      <dt>&lt;top-options&gt;</dt>
      <dd>
        <strong>Syntax:</strong>countfield=&lt;string&gt; | limit=&lt;int&gt; | otherstr=&lt;string&gt; | percentfield=&lt;string&gt; | showcount=&lt;bool&gt; | showperc=&lt;bool&gt; | useother=&lt;bool&gt;</dd>
      <dd>
        <strong>Description:</strong>Options for the<code>top</code>command. See<strong>Top options</strong>.</dd>
    </dl>
    <dl>
      <dt>&lt;by-clause&gt;</dt>
      <dd>
        <strong>Syntax:</strong>BY &lt;field-list&gt;</dd>
      <dd>
        <strong>Description:</strong>The name of one or more fields to group by.</dd>
    </dl>
    <h3>
      <span id="Top_options">Top options</span>
    </h3>
    <dl>
      <dt>countfield</dt>
      <dd>
        <strong>Syntax:</strong>countfield=&lt;string&gt;</dd>
      <dd>
        <strong>Description:</strong>The name of a new field that the value of count is written to.</dd>
      <dd>
        <strong>Default:</strong>"count"</dd>
    </dl>
    <dl>
      <dt>limit</dt>
      <dd>
        <strong>Syntax:</strong>limit=&lt;int&gt;</dd>
      <dd>
        <strong>Description:</strong>Specifies how many tuples to return, "0" returns all values.</dd>
      <dd>
        <strong>Default:</strong>"10"</dd>
    </dl>
    <dl>
      <dt>otherstr</dt>
      <dd>
        <strong>Syntax:</strong>otherstr=&lt;string&gt;</dd>
      <dd>
        <strong>Description:</strong>If useother is true, specify the value that is written into the row representing all other values.</dd>
      <dd>
        <strong>Default:</strong>"OTHER"</dd>
    </dl>
    <dl>
      <dt>percentfield</dt>
      <dd>
        <strong>Syntax:</strong>percentfield=&lt;string&gt;</dd>
      <dd>
        <strong>Description:</strong>Name of a new field to write the value of percentage.</dd>
      <dd>
        <strong>Default:</strong>"percent"</dd>
    </dl>
    <dl>
      <dt>showcount</dt>
      <dd>
        <strong>Syntax:</strong>showcount=&lt;bool&gt;</dd>
      <dd>
        <strong>Description:</strong>Specify whether to create a field called "count" (see "countfield" option) with the count of that tuple.</dd>
      <dd>
        <strong>Default:</strong>true</dd>
    </dl>
    <dl>
      <dt>showperc</dt>
      <dd>
        <strong>Syntax:</strong>showperc=&lt;bool&gt;</dd>
      <dd>
        <strong>Description:</strong>Specify whether to create a field called "percent" (see "percentfield" option) with the relative prevalence of that tuple.</dd>
      <dd>
        <strong>Default:</strong>true</dd>
    </dl>
    <dl>
      <dt>useother</dt>
      <dd>
        <strong>Syntax:</strong>useother=&lt;bool&gt;</dd>
      <dd>
        <strong>Description:</strong>Specify whether or not to add a row that represents all values not included due to the limit cutoff.</dd>
      <dd>
        <strong>Default</strong>: false</dd>
    </dl>
    <h2></h2>

    <a href="http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Top" target="_blank">
      Search Reference Guide</a>

  </html>
</panel>

top.xml

<dashboard>
  <label>top</label>
  <row>
    <panel ref="top_desc" app="spl_examples"></panel>
  </row>
  <row>
    <panel>
      <html>
        <p>
          <b>Example 1. Return the 20 most common values for a field</b>
        </p>
        <p>This search returns the 20 most common values of the "referer" field. The results show the number of events (count) that have that a count of referer, and the percent that each referer is of the total number of events.</p>
        <b>Search:</b> index="splexamples" sourcetype="access_combined_wcookie"  | top limit=20 referer
      </html>
      <table>
        <search>
          <query>index="splexamples" sourcetype="access_combined_wcookie"  | top limit=20 referer</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="count">10</option>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">cell</option>
        <option name="percentagesRow">false</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <html>
        <p>
          <b>Example 2. Return top values for one field organized by another field</b>
        </p>
        <p>This search returns the top "action" values for each "referer_domain".</p>
        <p><b>Search:</b> index="splexamples" sourcetype="access_combined_wcookie"  | top action by referer_domain</p>
        <p>Because a limit is not specified, this returns all the combinations of values for "action" and "referer_domain" as well as the counts and percentages:</p>
      </html>
      <table>
        <search>
          <query>index="splexamples" sourcetype="access_combined_wcookie"  | top action by referer_domain</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="count">10</option>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">cell</option>
        <option name="percentagesRow">false</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <html>
        <p>
          <b>Example 3. Returns the top product purchased for each category</b>
        </p>
        <p>This search returns the top product purchased for each category. Do not show the percent field. Rename the count field to "total".</p>
        <b>Search:</b> index="splexamples" sourcetype="access_combined_wcookie"  status=200 action=purchase | top 1 productName by categoryId showperc=f countfield=total
      </html>
      <table>
        <search>
          <query>index="splexamples" sourcetype="access_combined_wcookie"  status=200 action=purchase | top 1 productName by categoryId showperc=f countfield=total</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="count">10</option>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">cell</option>
        <option name="percentagesRow">false</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
      </table>
    </panel>
  </row>
</dashboard>

top.xmlがtop_desc.xmlを呼び出して、一つのダッシュボードにしている。
これが各コマンド分ある。

eventgenがイベントを作ると、サンプル結果が上手く表示される（みたい。）

https://www.splunk.com/en_us/blog/tips-and-tricks/introducing-simdata-v1-2.html
eventgenもこっちに変わるのかな？

別なappsが必要なよりも、チュートリアルデータでやったほうがいいかもしれない。

https://qiita.com/maroon/items/86412cbeee156899a5a7 で紹介もされているし。

まとめ

https://docs.splunk.com/Documentation/Splunk/8.0.5/SearchReference/Commandsbycategory
だと157コマンド

まだまだメンバーが足りないので是非是非
gitの使い方覚えないと