splunklib.client など

splunklib.client など使い方

注目キーワード
CLASS - splunklib.client.Job(service, sid, **kwargs)
job.is_done()
results.ResultsReader(job.results())

import splunklib.client as client
import splunklib.results as results
from time import sleep
service = client.connect(...)
job = service.jobs.create("search * | head 5")
while not job.is_done():
sleep(.2)
rr = results.ResultsReader(job.results())
for result in rr:
if isinstance(result, results.Message):
# Diagnostic messages may be returned in the results
print '%s: %s' % (result.type, result.message)
elif isinstance(result, dict):
# Normal events are returned as dicts
print result
assert rr.is_preview == False

#サーチ実行の関数の定義 (コピペしてください）

def execute_query(searchquery_normal,
kwargs_normalsearch={"exec_mode": "normal"},
kwargs_options={"output_mode": "csv", "count": 100000}):

# Execute Search
job = service.jobs.create(searchquery_normal, **kwargs_normalsearch)

# サーチが完了するまで、ポールするようにする。
while True:
    while not job.is_ready():
        pass
    stats = {"isDone": job["isDone"], "doneProgress": float(job["doneProgress"])*100, 
             "scanCount": int(job["scanCount"]), "eventCount": int(job["eventCount"]), 
             "resultCount": int(job["resultCount"])}
    status = ("\r%(doneProgress)03.1f%%   %(scanCount)d scanned " 
              "%(eventCount)d matched   %(resultCount)d results") % stats

    sys.stdout.write(status)
    sys.stdout.flush()
    if stats["isDone"] == "1":
        sys.stdout.write("\nDone!")
        break
    time.sleep(0.5)

# サーチ結果を取得＆リターン
csv_results = job.results(**kwargs_options).read()
job.cancel()
return csv_results

jobsについてcreate, deleteについて

When creating jobs through client there is no way to get the information about any jobs that are spawned by commands like append. This information is important as these jobs might need to be deleted and the information they give back is certainly helpful.

splunklib.client is excellent for running many of the same search with different time. As a user I would like to delete my searches (and the sub searches they create) to not take up too much disk space and exceed my limit.

import splunklib.client as client
service=client.connect(...)
...
search="| search index=bla | append [ | makeresults bla] " (two searches are created here)
...
jobs=service.jobs
job=jobs.create("search", **kwargs)
...
jobs.delete(job["sid"]) (only one is deleted as the appended search has a different sid)

Which is a problem if I am making hundreds of these with a while loop.

The sid of the appended searches does have the sid of the parent in it as well as their own unique identifiers. They should be easy to find and return.

splunklib.resultsについて

splunklib.results
The splunklib.results module provides a streaming XML reader for Splunk search results.

Splunk search results can be returned in a variety of formats including XML, JSON, and CSV. To make it easier to stream search results in XML format, they are returned as a stream of XML fragments, not as a single XML document. This module supports incrementally reading one result record at a time from such a result stream. This module also provides a friendly iterator-based interface for accessing search results while avoiding buffering the result set, which can be very large.

To use the reader, instantiate JSONResultsReader on a search result stream as follows::

reader = ResultsReader(result_stream)
for item in reader:
print(item)
print "Results are a preview: %s" % reader.is_preview
class splunklib.results.Message(type_, message)
This class represents informational messages that Splunk interleaves in the results stream.

Message takes two arguments: a string giving the message type (e.g., “DEBUG”), and a string giving the message itself.

Example:

m = Message("DEBUG", "There's something in that variable...")
class splunklib.results.JSONResultsReader(stream)
This class returns dictionaries and Splunk messages from a JSON results stream. JSONResultsReader is iterable, and returns a dict for results, or a Message object for Splunk messages. This class has one field, is_preview, which is True when the results are a preview from a running search, or False when the results are from a completed search.

This function has no network activity other than what is implicit in the stream it operates on.

Parameters: stream – The stream to read from (any object that supports.read()).
Example:

import results
response = ... # the body of an HTTP response
reader = results.JSONResultsReader(response)
for result in reader:
if isinstance(result, dict):
print "Result: %s" % result
elif isinstance(result, results.Message):
print "Message: %s" % result
print "is_preview = %s " % reader.is_preview