はじめに
最近話題のJevを使ってみました。Elasticのサーチ、オブザーバビリティ、セキュリティのソリューションと非常に相性が良さそうです。
今回は面白半分に、Jev公式ドキュメンテーションに載っているSmart home assistant demoと同じようなアプリケーションをElastic StackのWorkflowsとDashboardで再現してみました。
（公式Demoのアプリのコードがまだ出ていないようでしたので、動画をみて見様見真似で再現してます。）
Typesafe(Jev) のSmart home assistant demo
https://docs.typesafe.ai/demos/smart-home
Pythonなどのプログラミング言語はElasticでは直接は使えませんが、Yaml形式で処理を書くElastic WorkflowsでもJevを使って色々面白いことができました。
TL;DR
Elastic Stack (Elastic Cloud Serverlessを使っています）で動く様子をGIF動画でご覧ください。チャットでの指示に応じてホームデバイスのステータスが変化します。
実はこのElastic Kibanaのダッシュボードもつい最近できるようになったカスタムHTMLパネルを使っています。これによってダッシュボードが見栄えします。
このダッシュボードもElastic内で使えるAIが一発で作ってくれました。（v9.4登場のKibana Dashbhoard APIが使われています。）
ポイント ... 何がJevなのか
各チャット指示において、どのデバイスに対して、何の変更をするのかをJevに判定してもらっています。
- Turn on all the lights
照明系デバイスが全部ONになりました - Turn off all the lights other than the bedroom
ベッドルーム以外の照明系デバイスが全部OFFになりました - I want to listen to music in my living room!
リビングのSpeakerがONになりました - Turn on the speaker in my Bedroom! (これは上の動画にはありません）
BedroomにはSpeakerはないので、正しく何も変更されませんでした - 寝室の照明消して (これは上の動画にはありません）
日本語の「寝室」と指定しても Jevで"Bedroom Light" のデバイスを識別してオフにしてくれました
今までのLLMでも頑張れば実現できなくはないです。しかしJevは以下のようなメリットがあります。
- 今までのLLMよりも料金がめちゃくちゃ安い （今回のDemo作成でたくさん使っても$0.0065でした）
- 速い
- 指示が正確に出しやすい。具体的な質問、具体的なstate(この場合デバイスや状態など）を与えるようなAPIの作りになっている
- 回答精度が高い。このデモの場合それぞれの質問に対して、デバイス毎に回答とその確度スコアがレスポンスとして返ってくる。（今までのLLMでは、確実にこの形式でレスポンスを返させるには苦労する。）
私のJev (Typesafe) 課金の様子
実装方法
Elastic Workflowsの作成
ElasticのWorkflowがスクリプトのような役割で、Jev APIにアクセスしたりダッシュボードに表示しているElasticsearch内のデータを更新したりしています。
実際のWorkflowはもっと複雑で、自力で書くに今回の場合大変すぎるので、Elasticで提供されているWorkflow生成のツールやスキルを使って以下のように指示しています。この前段に公式Demo動画のスクショをClaudeに与え、Demo構成物を予めテキスト化しています。
Below is information of a Typesafe AI demo. By selecting the order prompts it uses Typesafe classifications to select and change the device and state.
Create a workflow that receives the order prompts as input and executes Typesafe api to select the device and state to change. Let me know if you need any clarification.
---
**Top bar (input & prompt examples):**
- Input field: `Get the coffee boiling`
- Button: `No Device Context ▼` | `Send`
**Prompt example chips (row 1):**
- Turn on the living room lights
- Turn off all the lights
- Turn on my kid's light
- Is the kitchen light on?
- Shut off all the music in the house
- Get the coffee boiling
- Let's get some outside music going
**Prompt example chips (row 2):**
- It's going to be a hot day — turn on all the fans
- Turn off the kitchen lights and lock the office door
- Lock up the whole house
- Set the bedroom to heat
- Who won the World Series in 1989?
**Prompt example chips (row 3):**
- Turn on the living room lights and turn off the kitchen. Oh, and can you get the coffee started?
---
**Left panel — Smart Home (1BR · 10 devices | Reset)**
**Living Room**
- Overhead Lights — On · 80%
- Floor Lamp — On · 50%
- Ceiling Fan — Off
- Smart Speaker — Off
**Kitchen**
- Kitchen Lights — Off
- Coffee Maker — On *(highlighted in red/active)*
**Bedroom**
- Bedroom Light — Off
- Thermostat — A/C · 72°F
**Office**
- Desk Lamp — Off
- Door Lock — Locked
---
**Right panel — Decision Trace**
> "Get the coffee boiling"
> **TypeSafe** 185ms
*(greyed out previous query: "Shut off all the music in the house" (729ms))*
| Tag | Question | Score |
|---|---|---|
| `+ choice` | What is the user's intent? | **0.98** |
| | `smarthome_command` 1.00 · `information_request` 0.00 · `smarthome_query` 0.00 | |
| `noul` | Does this request contain multiple commands? | **0.03** |
| | 2.8% probability | |
| `+ choice` | What is the scope of the request? | **0.97** |
| | `specific_device` 1.00 · `area` 0.00 · `whole_house` 0.00 | |
| `+ choice` | What kind of device is being targeted? | **0.99** |
| | `appliance` 1.00 · `thermostat` 0.00 · `lock` 0.00 · `speaker` 0.00 · `fan` 0.00 · `light` 0.00 | |
| `+ choice` | Which room is the user referring to? | **0.71** |
| | `kitchen` 0.53 · `none_of_these` 0.16 · `living_room` 0.09 · `office` 0.09 · `bedroom` 0.09 | |
| `+ choice` | Which specific device should receive the command? | **0.99** |
| | `kitchen_coffee_maker` 1.00 · `none_of_these` 0.00 · `living_room_lamp` 0.00 · `living_room_fan` 0.00 · `office_lock` 0.00 · `kitchen_light` 0.00 · `living_room_overhead` 0.00 · `office_light` 0.00 · `bedroom_light` 0.00 · `bedroom_thermostat` 0.00 · `living_room_speaker` 0.00 | |
| `+ choice` | What should happen to the lights? | |
| | `turn_on` 0.97 · `turn_off` 0.20 · `dim` 0.12 | |
| `+ choice` | What should happen to the fans? | |
| | `turn_off` 1.0 | |
Elastic WorkflowのYaml
上記のチャット指示である程度のベースのYamlは作ってくれましたが、一発で完璧に動くものとはならないので、その後修正を施してできあがったDemoコードが以下のものです。
まずは全体のWorkflow Stepの様子。赤枠がJev(SmartType) APIを使っているところ。
実際のWorkflow Yamlはこちら（長いので畳んでいます)
name: Smart Home TypeSafe Classifier
enabled: true
description: >
Classifies a natural-language smart home prompt using the TypeSafe speculative
fan-out API (all questions in one call), then resolves the target device and
action via if/else logic, producing a structured decision object.
Supports multi-command prompts by splitting them into atomic sub-commands and
classifying each one independently, producing a decisions array.
triggers:
- type: manual
inputs:
properties:
prompt:
type: string
description: Natural language smart home command or query (e.g. "Turn off the kitchen light")
api_key:
type: string
description: TypeSafe API key used for Bearer authentication
required:
- prompt
- api_key
steps:
# ─────────────────────────────────────────────
# 1. Call TypeSafe — speculative fan-out
# ─────────────────────────────────────────────
- name: classify
type: http
with:
url: https://api.typesafe.ai/v1/systemone
method: POST
headers:
Authorization: "Bearer {{ inputs.api_key }}"
Content-Type: application/json
body:
state: "{{ inputs.prompt }}"
model: jev-latest
questions:
intent:
type: choice
instructions: What is the user's intent?
criteria:
smarthome_command: The user wants to control a smart home device
information_request: The user is asking for information
smarthome_query: The user is querying the state of a smart home device
multi_command:
type: noul
instructions: Does this request contain multiple commands?
scope:
type: choice
instructions: What is the scope of the request?
criteria:
specific_device: Targeting one specific device
area: Targeting all devices in a room/area
whole_house: Targeting devices across the entire house
device_type:
type: choice
instructions: What kind of device is being targeted?
criteria:
appliance: A household appliance (e.g. coffee maker, dishwasher)
thermostat: A temperature-control device
lock: A door lock
speaker: An audio speaker
fan: A ceiling or stand fan
light: A light fixture or lamp
room:
type: choice
instructions: Which room is the user referring to?
criteria:
kitchen: The kitchen
living_room: The living room
bedroom: The bedroom
office: The home office
none_of_these: None of the above / not specified
target_device:
type: choice
instructions: Which specific device should receive the command?
criteria:
kitchen_coffee_maker: Coffee maker in the kitchen
kitchen_light: Light in the kitchen
living_room_overhead: Overhead light in the living room
living_room_lamp: Lamp in the living room
living_room_fan: Fan in the living room
living_room_speaker: Speaker in the living room
bedroom_light: Light in the bedroom
bedroom_thermostat: Thermostat in the bedroom
office_lock: Lock on the office door
office_light: Light in the office
none_of_these: None of these / not specified
action:
type: choice
instructions: >
Given the device type already identified for this request, what should the
device's final state be set to? For lights, fans, appliances, and speakers,
choose "on" to turn on, dim, or brighten the device, or "off" to turn it off.
For locks, choose "locked" to lock the door or "unlocked" to unlock it.
For thermostats, choose "heat" to set heat mode or "cool" to set cool mode.
criteria:
"on": Turn on, dim, or brighten the device (light, fan, appliance, or speaker)
"off": Turn off the device (light, fan, appliance, speaker, or thermostat)
locked: Lock the door
unlocked: Unlock the door
heat: Set the thermostat to heat mode
cool: Set the thermostat to cool mode
# ─────────────────────────────────────────────
# 2. Branch: multi-command vs. single-command
# ───────────────────────��─────────────────────
- name: check_multi_command
type: if
condition: "steps.classify.output.data.answers.multi_command.noul > 0.5"
# ══════════════════════════════════════════
# MULTI-COMMAND PATH
# ══════════════════════════════════════════
steps:
# 2a. Ask Claude Haiku to split the prompt into atomic sub-commands
- name: split_commands
type: ai.prompt
connector-id: Anthropic-Claude-Haiku-4-5
with:
systemPrompt: >
You are a smart home assistant. Your task is to decompose a user's
natural-language request into individual, atomic smart home commands.
Each command must target exactly one device and one action.
Return ONLY the JSON object — no explanation, no markdown fences.
prompt: >
Split the following user request into individual atomic smart home
commands. Each command should be a short, self-contained instruction
targeting a single device.
User request: "{{ inputs.prompt }}"
schema:
type: object
properties:
commands:
type: array
description: List of atomic smart home command strings
items:
type: string
required:
- commands
# 2b. Loop over each atomic sub-command
- name: process_sub_commands
type: foreach
foreach: "{{ steps.split_commands.output.content.commands | json }}"
steps:
# 2c-i. Re-classify each sub-command with the full TypeSafe fan-out
- name: classify_sub
type: http
with:
url: https://api.typesafe.ai/v1/systemone
method: POST
headers:
Authorization: "Bearer {{ inputs.api_key }}"
Content-Type: application/json
body:
state: "{{ foreach.item }}"
model: jev-latest
questions:
intent:
type: choice
instructions: What is the user's intent?
criteria:
smarthome_command: The user wants to control a smart home device
information_request: The user is asking for information
smarthome_query: The user is querying the state of a smart home device
multi_command:
type: noul
instructions: Does this request contain multiple commands?
scope:
type: choice
instructions: What is the scope of the request?
criteria:
specific_device: Targeting one specific device
area: Targeting all devices in a room/area
whole_house: Targeting devices across the entire house
device_type:
type: choice
instructions: What kind of device is being targeted?
criteria:
appliance: A household appliance (e.g. coffee maker, dishwasher)
thermostat: A temperature-control device
lock: A door lock
speaker: An audio speaker
fan: A ceiling or stand fan
light: A light fixture or lamp
room:
type: choice
instructions: Which room is the user referring to?
criteria:
kitchen: The kitchen
living_room: The living room
bedroom: The bedroom
office: The home office
none_of_these: None of the above / not specified
target_device:
type: choice
instructions: Which specific device should receive the command?
criteria:
kitchen_coffee_maker: Coffee maker in the kitchen
kitchen_light: Light in the kitchen
living_room_overhead: Overhead light in the living room
living_room_lamp: Lamp in the living room
living_room_fan: Fan in the living room
living_room_speaker: Speaker in the living room
bedroom_light: Light in the bedroom
bedroom_thermostat: Thermostat in the bedroom
office_lock: Lock on the office door
office_light: Light in the office
none_of_these: None of these / not specified
action:
type: choice
instructions: >
Given the device type already identified for this request, what should the
device's final state be set to? For lights, fans, appliances, and speakers,
choose "on" to turn on, dim, or brighten the device, or "off" to turn it off.
For locks, choose "locked" to lock the door or "unlocked" to unlock it.
For thermostats, choose "heat" to set heat mode or "cool" to set cool mode.
criteria:
"on": Turn on, dim, or brighten the device (light, fan, appliance, or speaker)
"off": Turn off the device (light, fan, appliance, speaker, or thermostat)
locked: Lock the door
unlocked: Unlock the door
heat: Set the thermostat to heat mode
cool: Set the thermostat to cool mode
# 2c-ii. Route each sub-command by intent
- name: route_sub
type: if
condition: "steps.classify_sub.output.data.answers.intent.choice : \"smarthome_command\""
steps:
- name: set_sub_decision
type: data.set
with:
sub_decision:
sub_command: "{{ foreach.item }}"
device: "{{ steps.classify_sub.output.data.answers.target_device.choice }}"
action: "{{ steps.classify_sub.output.data.answers.action.choice }}"
room: "{{ steps.classify_sub.output.data.answers.room.choice }}"
scope: "{{ steps.classify_sub.output.data.answers.scope.choice }}"
device_type: "{{ steps.classify_sub.output.data.answers.device_type.choice }}"
else:
- name: set_sub_decision_non_command
type: data.set
with:
sub_decision:
sub_command: "{{ foreach.item }}"
device: null
action: null
room: "{{ steps.classify_sub.output.data.answers.room.choice }}"
scope: "{{ steps.classify_sub.output.data.answers.scope.choice }}"
device_type: "{{ steps.classify_sub.output.data.answers.device_type.choice }}"
intent: "{{ steps.classify_sub.output.data.answers.intent.choice }}"
# ── Write state back to smart-home-devices index ───────────────────
- name: update_sub_device_state
type: elasticsearch.bulk
with:
index: smart-home-devices
operations:
- update:
_id: "{{ steps.classify_sub.output.data.answers.target_device.choice }}"
- doc:
state: "{{ steps.classify_sub.output.data.answers.action.choice }}"
last_updated: "{{ now | date_to_xmlschema }}"
# 2d. Collect all per-iteration sub_decisions into a decisions array
- name: collect_decisions
type: data.set
with:
decisions: "{{ steps.process_sub_commands.output | map: 'sub_decision' }}"
# 2e. Log a summary of all sub-command decisions
- name: log_decisions
type: console
with:
message: |
╔══════════════════════════════════════════╗
Smart Home Multi-Command Summary
╚══════════════════════════════════════════╝
Original Prompt : {{ inputs.prompt }}
Multi-Command : YES (noul={{ steps.classify.output.data.answers.multi_command.noul }})
─────────────────────────────────────────
{%- for d in steps.collect_decisions.output.decisions %}
[{{ forloop.index }}] {{ d.sub_command }} → {{ d.action }} on {{ d.device }} (room: {{ d.room }})
{%- endfor %}
─────────────────────────────────────────
Model used : {{ steps.classify.output.data.model }}
# ══════════════════════════════════════════
# SINGLE-COMMAND PATH
# ══════════════════════════════════════════
else:
# ── Route: is this a smarthome_command? ───
- name: route_intent
type: if
condition: "steps.classify.output.data.answers.intent.choice : \"smarthome_command\""
steps:
# Build the decision object directly from the unified action answer
- name: set_decision
type: data.set
with:
decision:
device: "{{ steps.classify.output.data.answers.target_device.choice }}"
action: "{{ steps.classify.output.data.answers.action.choice }}"
room: "{{ steps.classify.output.data.answers.room.choice }}"
scope: "{{ steps.classify.output.data.answers.scope.choice }}"
device_type: "{{ steps.classify.output.data.answers.device_type.choice }}"
raw_answers: "{{ steps.classify.output.data.answers | json }}"
# ── Dispatch: known device vs. ambiguous/whole-house scope ─────────
- name: check_target_device
type: if
condition: "NOT steps.classify.output.data.answers.target_device.choice : none_of_these"
# ── KNOWN DEVICE — direct update ──────────────────────────────
steps:
- name: update_known_device
type: elasticsearch.bulk
with:
index: smart-home-devices
operations:
- update:
_id: "{{ steps.classify.output.data.answers.target_device.choice }}"
- doc:
state: "{{ steps.classify.output.data.answers.action.choice }}"
last_updated: "{{ now | date_to_xmlschema }}"
# ── AMBIGUOUS / WHOLE-HOUSE — resolve via TypeSafe Noul fan-out (multi-select) ──
else:
# ── Bug 1 fix: pre-build proper term objects for must clause ────
- name: build_must_clause
type: data.set
with:
must_device_type:
term:
device_type: "{{ steps.classify.output.data.answers.device_type.choice }}"
must_room:
term:
room: "{{ steps.classify.output.data.answers.room.choice }}"
# ── Bug 1 fix: split fetch into area vs. global scope ────────
- name: fetch_candidate_devices_check_scope
type: if
condition: "steps.classify.output.data.answers.scope.choice : area"
# area scope — filter by both device_type AND room
steps:
- name: fetch_candidate_devices_area
type: elasticsearch.search
with:
index: smart-home-devices
size: 20
_source:
- name
- room
- device_type
query:
bool:
must:
- term:
device_type: "{{ steps.classify.output.data.answers.device_type.choice }}"
- term:
room: "{{ steps.classify.output.data.answers.room.choice }}"
# non-area scope — filter by device_type only
else:
- name: fetch_candidate_devices_global
type: elasticsearch.search
with:
index: smart-home-devices
size: 20
_source:
- name
- room
- device_type
query:
bool:
must:
- term:
device_type: "{{ steps.classify.output.data.answers.device_type.choice }}"
# ── Consolidate hits from whichever branch ran ────────────────
- name: merge_fetch_results
type: data.set
with:
hits: "${{ steps.fetch_candidate_devices_area.output.hits.hits | default: steps.fetch_candidate_devices_global.output.hits.hits }}"
# Step 2 — Build a structured candidates array from the ES hits
- name: build_device_candidates
type: data.set
with:
candidates: "{{ steps.merge_fetch_results.output.hits | map: '_source' }}"
candidate_ids: "{{ steps.merge_fetch_results.output.hits | map: '_id' }}"
candidates_json: >-
{%- liquid
assign arr = "["
for hit in steps.merge_fetch_results.output.hits
if forloop.first == false
assign arr = arr | append: ","
endif
assign entry = '{"id":"' | append: hit._id | append: '","name":"' | append: hit._source.name | append: '","room":"' | append: hit._source.room | append: '","device_type":"' | append: hit._source.device_type | append: '"}'
assign arr = arr | append: entry
endfor
assign arr = arr | append: "]"
echo arr
-%}
# Step 3a — Dynamically build the TypeSafe request body: one Noul
# question per candidate device ("is_match_<candidate_id>"), each
# thresholded independently, following the TypeSafe Noul cookbook
# pattern for selecting multiple items in a single request.
- name: build_resolve_request_body
type: data.set
with:
resolve_request_body: >-
{%- liquid
assign command_escaped = inputs.prompt | replace: '"', '\"'
assign body = '{"model":"jev-latest","state":{"command":"' | append: command_escaped | append: '","scope":"' | append: steps.classify.output.data.answers.scope.choice | append: '","room":"' | append: steps.classify.output.data.answers.room.choice | append: '","device_type":"' | append: steps.classify.output.data.answers.device_type.choice | append: '","action":"' | append: steps.classify.output.data.answers.action.choice | append: '","candidates":' | append: steps.build_device_candidates.output.candidates_json | append: '},"questions":{'
assign first = true
for hit in steps.merge_fetch_results.output.hits
if first == false
assign body = body | append: ","
endif
assign q_key = "is_match_" | append: hit._id
assign instructions = "Given the user command '" | append: inputs.prompt | append: "', should this candidate device (" | append: hit._source.name | append: " in " | append: hit._source.room | append: ", a " | append: hit._source.device_type | append: ") be turned " | append: steps.classify.output.data.answers.action.choice | append: "?"
assign q_entry = '"' | append: q_key | append: '":{"type":"noul","instructions":"' | append: instructions | append: '"}'
assign body = body | append: q_entry
assign first = false
endfor
assign body = body | append: "}}"
echo body
-%}
# Step 3b — Send the dynamically-built request: a single TypeSafe
# call carrying one independently-thresholded Noul question per
# candidate, instead of a single Choice with a fixed winner.
- name: resolve_target_devices
type: http
with:
url: https://api.typesafe.ai/v1/systemone
method: POST
headers:
Authorization: "Bearer {{ inputs.api_key }}"
Content-Type: application/json
body: "{{ steps.build_resolve_request_body.output.resolve_request_body }}"
# Step 4 — Determine which candidate ids cleared the match
# threshold (noul > 0.6), independently for each candidate.
- name: compute_matched_device_ids
type: data.set
with:
matched_ids_json: >-
{%- liquid
assign arr = "["
assign first = true
for hit in steps.merge_fetch_results.output.hits
assign q_key = "is_match_" | append: hit._id
assign answer = steps.resolve_target_devices.output.data.answers[q_key]
assign score = answer.noul
if score > 0.6
if first == false
assign arr = arr | append: ","
endif
assign arr = arr | append: '"' | append: hit._id | append: '"'
assign first = false
endif
endfor
assign arr = arr | append: "]"
echo arr
-%}
# Step 5 — Parse the matched id list back into a structured array
- name: parse_matched_device_ids
type: data.parseJson
source: "{{ steps.compute_matched_device_ids.output.matched_ids_json }}"
# Step 6 — Update EVERY device whose noul cleared the threshold
# (zero matches → zero iterations → no update, same as before).
- name: update_matching_devices
type: foreach
foreach: "{{ steps.parse_matched_device_ids.output | json }}"
steps:
# ── Bug fix: capture foreach.item via data.set before using it
# as the bulk operation's _id — direct interpolation of
# foreach.item inside the nested elasticsearch.bulk operations
# array fails with "id is missing".
- name: build_matched_op_id
type: data.set
with:
op_id: "{{ foreach.item }}"
- name: update_matched_device_state
type: elasticsearch.bulk
with:
index: smart-home-devices
operations:
- update:
_id: "{{ steps.build_matched_op_id.output.op_id }}"
- doc:
state: "{{ steps.classify.output.data.answers.action.choice }}"
last_updated: "{{ now | date_to_xmlschema }}"
else:
# Non-command intent (query or information request)
- name: set_decision_non_command
type: data.set
with:
decision:
device: null
action: null
room: "{{ steps.classify.output.data.answers.room.choice }}"
scope: "{{ steps.classify.output.data.answers.scope.choice }}"
device_type: "{{ steps.classify.output.data.answers.device_type.choice }}"
intent: "{{ steps.classify.output.data.answers.intent.choice }}"
raw_answers: "{{ steps.classify.output.data.answers }}"
# Log the final single-command decision
- name: log_decision
type: console
with:
message: |
╔══════════════════════════════════════════╗
Smart Home Decision Summary
╚══════════════════════════════════════════╝
Prompt : {{ inputs.prompt }}
Intent : {{ steps.classify.output.data.answers.intent.choice }}
Scope : {{ steps.classify.output.data.answers.scope.choice }}
Room : {{ steps.classify.output.data.answers.room.choice }}
Device Type : {{ steps.classify.output.data.answers.device_type.choice }}
Target Dev : {{ steps.classify.output.data.answers.target_device.choice }}
Multi-cmd : {{ steps.classify.output.data.answers.multi_command.noul }}
─────────────────────────────────────────
Action : {{ steps.classify.output.data.answers.action.choice }} → {{ steps.classify.output.data.answers.target_device.choice }} (room: {{ steps.classify.output.data.answers.room.choice }}, scope: {{ steps.classify.output.data.answers.scope.choice }})
─────────────────────────────────────────
Model used : {{ steps.classify.output.data.model }}
おわり
Jevの登場で人のIntent意図をシステムに正確に落とし込めるようになりました。
オブザーバビリティやセキュリティでも利用価値が非常にありそうなので、ぜひElastic + Jevを使ってみてください。