0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

Run a Local Search Engine and Text Analysis in Google Colab with nlp4j-local-search 0.5.0

0
Posted at

Run a Local Search Engine and Text Analysis in Google Colab with nlp4j-local-search 0.5.0

nlp4j-local-search is a lightweight Python library that lets you use Apache Lucene without running Elasticsearch, OpenSearch, Solr, or Docker.

Version 0.5.0 was released on PyPI on August 23, 2026. ([PyPI][1])

It is useful for small search experiments, NLP experiments, Jupyter Notebook, and Google Colab.

Internally, the library uses Java and Apache Lucene, but you can use it from a simple Python API.

In this article, we will try two things:

  1. Build a small full-text search engine
  2. Analyze characteristic words with view()

Both examples can be run easily in Google Colab.

Installation

In Google Colab, run:

!pip install -q nlp4j-local-search==0.5.0

Then import SearchEngine:

from nlp4j_local_search import SearchEngine

That's all we need to get started.

Example 1: Full-Text Search

Let's create a very small English search engine.

from nlp4j_local_search import SearchEngine

with SearchEngine("en") as engine:

    engine.add(
        "1",
        "Developers are searching documents with a local search engine."
    )

    engine.add(
        "2",
        "A developer searched many documents yesterday."
    )

    engine.add(
        "3",
        "This tool searches local JSON documents."
    )

    engine.commit()

    results = engine.search("search", limit=10)

    for r in results:
        print(r.id, r.body)

SearchEngine("en") uses English text analysis, so different forms such as:

search
searched
searches
searching

can be handled by the search engine. ([PyPI][1])

There is no need to start a separate search server.

The search index is created locally and can be used directly from Python.

Example 2: Text Analysis with view()

Version 0.5.0 can also be used for simple text analysis.

For example, suppose we have the following documents:

from nlp4j_local_search import SearchEngine

DOCUMENTS = [
    ("1",  "The cat sat on the mat"),
    ("2",  "The cat chased the mouse"),
    ("3",  "The dog ran across the park"),
    ("4",  "The dog barked at the cat"),
    ("5",  "A mouse hid under the table"),
    ("6",  "The park has many trees and flowers"),
    ("7",  "Children played in the park"),
    ("8",  "The dog and the cat are friends"),
    ("9",  "She saw a mouse in the kitchen"),
    ("10", "He walked his dog in the park"),
]

with SearchEngine("en") as engine:

    for doc_id, body in DOCUMENTS:
        engine.add(doc_id, body)

    engine.commit()

    result = engine.view(
        query_field="word.noun",
        query_value="cat",
        field="word.noun",
        size=100,
    )

    print(
        "Target documents:",
        result.count,
        "/",
        result.total_count
    )

    for bucket in result.buckets:
        print(
            bucket.key,
            bucket.count,
            bucket.all_count,
            f"{bucket.relative_rate:.4f}",
        )

A result looks like this:

Target documents: 4 / 10

cat    4 4 2.5000
mat    1 1 2.5000
dog    2 4 1.2500
mouse  1 3 0.8333

This example is based on the 0.5.0 view() usage example.

What Is relative_rate?

view() compares a selected group of documents with the entire document collection.

Conceptually:

relative_rate
    = term rate in target documents
      / term rate in all documents

For example, we selected documents containing the noun cat.

There are:

4 target documents
10 documents in total

The noun cat appears in all four target documents, while it appears in four documents in the whole collection.

Therefore its relative_rate is:

(4 / 4) / (4 / 10)
= 2.5

A value greater than 1.0 means that the term appears relatively more often in the selected document group.

This calculation is performed by the underlying LocalAnalytics implementation; Python receives the result through the AnalyticsResult object.

Analyzing Verbs

We can also change the analyzed field.

For example, we can look at verbs that are characteristic of documents containing dog:

result = engine.view(
    query_field="word.noun",
    query_value="dog",
    field="word.verb",
    size=100,
)

for bucket in result.buckets:
    print(
        bucket.key,
        bucket.count,
        bucket.all_count,
        f"{bucket.relative_rate:.4f}",
    )

Example:

bark    1 1 2.5000
friend  1 1 2.5000
run     1 1 2.5000
walk    1 1 2.5000

The same SearchEngine can therefore be used not only for retrieving documents, but also for exploring the words that characterize a particular subset of documents.

Why Google Colab?

This type of library is especially convenient for notebook-based experiments.

You do not need to prepare:

Elasticsearch
OpenSearch
Solr
Docker
a search server

You can simply install the package, add some text, and start searching.

The project documentation specifically lists Jupyter Notebook and Google Colab among its intended use cases. ([PyPI][1])

For example, a complete Colab experiment can start with:

!pip install -q nlp4j-local-search==0.5.0

from nlp4j_local_search import SearchEngine

and then immediately proceed to:

engine.search(...)

or:

engine.view(...)

This makes it easy to experiment with both information retrieval and text analysis in the same Python environment.

Summary

With nlp4j-local-search 0.5.0, we can now perform tasks such as:

  • local full-text search
  • English and Japanese text search
  • field filtering
  • aggregation
  • vector search with user-provided vectors
  • OpenSearch-style Query DSL
  • characteristic-term analysis with view()

The basic idea remains simple:

from nlp4j_local_search import SearchEngine

with SearchEngine("en") as engine:
    engine.add("1", "some text")
    engine.commit()

    results = engine.search("text")

And when we want to explore the indexed text:

result = engine.view(
    query_field="word.noun",
    query_value="cat",
    field="word.noun",
)

No external search server is required.

For small NLP experiments, search prototypes, and Google Colab notebooks, this provides a simple way to use Lucene-based search and text analysis directly from Python.


0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?