Field names prefixed with underscores are omitted from latest transformsedit
If you use the latest type of transform and the source index has field names that start with an underscore (_) character, they are assumed to be internal fields. Those fields are omitted from the documents in the destination index.

また、retention_policyを指定できる。

retention_policy
(Optional, object) Defines a retention policy for the transform. Data that meets the defined criteria is deleted from the destination index.

了解可能な仕様ではあるが、これを組み合わせてretention_policyでアンダースコアのついたフィールドを指定し、dest_indexにインデックスされないフィールドでretentionを制御しようとすると機能しなかった。

アンダースコア付きの"_timestamp"フィールドで試す

PUT test_transform_retention
{
  "mappings": {
    "properties": {
      "_timestamp": {"type": "date"},
      "foo": {"type": "keyword"}
    }
  }
}
POST test_transform_retention/_doc
{
  "_timestamp": "2000-01-02",
  "foo":"A"
}
POST test_transform_retention/_doc
{
  "_timestamp": "2000-01-01",
  "foo":"A"
}
POST test_transform_retention/_doc
{
  "_timestamp": "2022-06-17",
  "foo":"B"
}
POST test_transform_retention/_doc
{
  "_timestamp": "2022-06-18",
  "foo":"B"
}

PUT _transform/transform_retention
{
  "source": {
    "index": "test_transform_retention"
  },
  "dest":{
    "index": "test_transform_retention_dest"
  },
  "latest":{
    "unique_key": "foo",
    "sort": "_timestamp"
  },
  "retention_policy":{
    "time":{
      "field": "_timestamp",
      "max_age": "365d"
    }
  }
}

POST _transform/transform_retention/_start

GET test_transform_retention_dest/_search

"hits" : [
      {
        "_index" : "test_transform_retention_dest",
        "_type" : "_doc",
        "_id" : "QRRx52klPRvG45a5oLgZ95sAAAAAAAAA",
        "_score" : 1.0,
        "_source" : {
          "foo" : "A"
        }
      },
      {
        "_index" : "test_transform_retention_dest",
        "_type" : "_doc",
        "_id" : "Qq7col5MOHvjTNMiAGonnqAAAAAAAAAA",
        "_score" : 1.0,
        "_source" : {
          "foo" : "B"
        }
      }
    ]

2件ともヒットしてしまう。

アンダースコアを排した"timestamp"フィールドで試す

"timestamp"フィールドに変更すると、1件のみインデックスされる。

PUT test_transform_retention2
{
  "mappings": {
    "properties": {
      "timestamp": {"type": "date"},
      "foo": {"type": "keyword"}
    }
  }
}

POST test_transform_retention/_doc
{
  "_timestamp": "2000-01-02",
  "foo":"A"
}
POST test_transform_retention/_doc
{
  "_timestamp": "2000-01-01",
  "foo":"A"
}
POST test_transform_retention/_doc
{
  "_timestamp": "2022-06-17",
  "foo":"B"
}
POST test_transform_retention/_doc
{
  "_timestamp": "2022-06-18",
  "foo":"B"
}

PUT _transform/transform_retention2
{
  "source": {
    "index": "test_transform_retention2"
  },
  "dest":{
    "index": "test_transform_retention_dest2"
  },
  "latest":{
    "unique_key": "foo",
    "sort": "timestamp"
  },
  "retention_policy":{
    "time":{
      "field": "timestamp",
      "max_age": "365d"
    }
  }
}

POST _transform/transform_retention2/_start

GET test_transform_retention_dest2/_search

"hits" : [
      {
        "_index" : "test_transform_retention_dest2",
        "_type" : "_doc",
        "_id" : "Qq7col5MOHvjTNMiAGonnqAAAAAAAAAA",
        "_score" : 1.0,
        "_source" : {
          "foo" : "B",
          "timestamp" : "2022-06-18"
        }
      }
    ]

Runtime Fieldも機能しない

PUT test_transform_retention_on_runtime
{
  "mappings": {
    "properties": {
      "timestamp": {"type": "date"},
      "runtimestamp":{
        "type": "date",
        "script":{
          "source": "emit(doc['timestamp'].value.toEpochMilli());"
        }
      },
      "foo": {"type": "keyword"}
    }
  }
}

POST test_transform_retention/_doc
{
  "timestamp": "2000-01-02",
  "foo":"A"
}
POST test_transform_retention/_doc
{
  "timestamp": "2000-01-01",
  "foo":"A"
}
POST test_transform_retention/_doc
{
  "timestamp": "2022-06-17",
  "foo":"B"
}
POST test_transform_retention/_doc
{
  "timestamp": "2022-06-18",
  "foo":"B"
}

GET test_transform_retention_on_runtime/_search
{
  "fields": ["*"]
}

PUT _transform/transform_retention_on_runtime
{
  "source": {
    "index": "test_transform_retention_on_runtime"
  },
  "dest":{
    "index": "test_transform_retention_on_runtime_dest"
  },
  "latest":{
    "unique_key": "foo",
    "sort": "runtimestamp"
  },
  "retention_policy":{
    "time":{
      "field": "runtimestamp",
      "max_age": "10d"
    }
  }
}

POST _transform/transform_retention_on_runtime/_start

GET _transform/transform_retention_on_runtime/_stats
GET test_transform_retention_on_runtime_dest/_search
{
  "fields": ["*"]
}

runtime fieldもretention_policyにおいては機能しない（sortとしては用いることができそう）。

したがって、retention_policyでは、dest indexでも実体のあるフィールドを指定する必要がありそうだ。