LoginSignup
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 3 years have passed since last update.

@T-ubu

Vector tile: Introduction to UNVT tools ( a kind of a tool catalog (not endorsed). As of April 2022)　

Posted at

The original article was written in Japanese (here) in April 2022. This article was translated and released in English in May 2022.

Introduction

The United Nations Vector Tile Toolkit (UNVT) was launched by @hfu in 2018 and is promoted under the UN Open GIS Initative. So far, various tools have been developed, which we can use them in each step of vector tile processing such as production, styling, hosting, optimization, and consumption (utilization).

In this article, I would like to introduce some of the UNVT tools that I use most often. I hope that it will help newbies of UNVT to have some idea on what tools are available. In addition, for those who work for vector tiles, I hope this article will serve as a kind of a catalog to show what UNVT can do.

Disclaimer

This article is based on my own experience as of April 2022. This article was first drafted in Japanese (here), then translated into English.

I am going to introduce each tool with its expected work area (production, styling, hosting, optimization, consumption, etc.), but it depends on my personal experiences and they may be used in different fields.

UNVT uses a lot of open source tools, so the tools introduced here should be called as a group of tools, and there are some overlaps among them. But I simply use "tool" in this article to simplify my explanation.

Tool list

Tools I use most often (They can be used for various purposes)

Tool name Release date (month) Main Developer(s) Areas of use Overview
unvt/tell 2021.9 hfu Use You can create a storytelling map using a vector tile map in the background (MapLibre library). As you scroll through the home page, the map scene moves and explanation box appears. Stories can be written in YAML files.
unvt/charites 2021.10 Geolonia Inc. Styles Vector map styles based on MapBox (and MapLibre) style specifications can be created/edited using easy-to-edit YAML files. We can also import style from a json style style file to YAML file, and vice versa. A live map preview function enable us to check our edits on the fly. A tool donated by Geolonia to UNVT.
unvt/kata 2022.3 GSI, Japan, Geolonia Inc. Production, Optimization It has the "filter" function to add layer name, maximum zoom level, minimum zoom level to each source file and output them to JSON sequence to be converted into vector tile. There is a "document" function that shows the layer name, attribute name, and zoom level range from the vector tile metadata. By passing the filtered data to Tippecanoe, you can easily create a vector tile with adjusted layer structure.
unvt/itoma 2022.3 GSI, Japan, Geolonia Inc. Styles, Hosting It displays vector tile map styles (YAML format imported by charites) in multiple vector tile libraries (Mapbox GL JS, MapLibre GL JS, ArcGIS API for Javascript) ) as preview by localhost. In addition, since it returns a response based on the VectorTileServer service of the ArcGIS REST API, you can check the use of vector tile maps in ArcGIS Online. (Note: Localhost needs to be https)
unvt / nanban 2020.4 hfu General Ubuntu / Intel based UNVT docker files. Give Windows users a Linux environment through a Docker container.
unvt / equinox 2020.3 hfu General UNVT related tool installer for Raspberry Pi.

There are more tools undre UNVT although they are not written here. For example, UNVT Portal is one of the tools and it was reported to UN Open GIS monthly meeting. I hope I can introduce them somewhere.)

Tools for advanced users (The usage will become a little more specialized)

Tool name Release date Developer (main) Areas of use Overview
unvt/onyx 2019 hfu Host A simple and scalable vector tile server. It is a nodejs/express server to host vector tiles and it delivers vector tiles in pbf format from mbtiles files. It can be executed in PM2.
unvt/marble 2021.12 ubukawa Hosting A server that extends the above mentioned onyx to deploy a function to return a response that conforms to the ArcGIS REST API VectorTileServer. It works with nodejs/express.
unvt/naru 2020 hfu General (Capacity Building) It works with Raspberry Pi and Docker. After downloading OSM, you can convert it to vector tiles, create styles using hocon, prepare sprites, and so on. It is a material for capacity building.
unvt/nsft 2022.2 ubukawa Hosting This is a workplace where I converted Noto Sans series fonts to PBF text fonts. It can be applicable to other fonts.
unvt/oz 2021.9 hfu Hosting A tool for server-side overzooming. Create large zoom level tiles from existing files.
unvt/zfxy-spec 2022.3 hfu General This is a draft specification for expanding zxy vector tile schema. It introduces the height index "f" to define a 3D tile space. We are paying attention to future development.

Tool introduction / Details (Selected tools from the above list)

I won't cover all of the tools above, but I will introduce some detailes of the selected tools. If you have any question or request about our tools, please let me know in the comments. I will reply as much as possible.

unvt/tell

overview:

You can create a storytelling map with a vector tile map in the background (MapLibre). As you scroll through the home page, the map scene moves and explanations appear. Stories can be written in YAML files.

  • You can create a map story by preparing an html file and a YAML file that specifies the location of the story and map. You can also use YAML tags in the html file.
  • This is a script developed as a workshop material and developed based on the scripts at https://github.com/optgeo/s (developed by hfu, taisukef, halsk).
  • This storytelling related work was inspired by mapbox, and developers explicitly express their appreciations to mapbox. thanks.

image.png

Repository URL:

Past efforts and references:

unvt/charites

overview:

Vector map styles based on MapBox (and MapLibre) style specifications can be created/edited using easy-to-edit YAML files. We can also import style from a json style style file to YAML file, and vice versa. A live map preview function enable us to check our edits on the fly. A tool donated by Geolonia to UNVT. (I'm a Windows user, so I often use this tool with unvt/nanban (a docker container).)

  • Create a YAML-formatted style from scratch, or import an existing style (JSON) into YAML format.
  • You can edit the style in YAML format.
  • Manage style in YAML files and classifies layers into groups to improve maintenance convenience and style readability.
  • Edited YAML files can be output in JSON format.
  • Real-time preview is also available, so you can see the result of your edit on the fly.

image.png
(As shown in this figure, one style information is managed by multiple YAML files.)

image.png
(Edit the YAML file while doing a live preview.)

Repository URL:

Past efforts and references:

unvt/kata

overview:

It has the "filter" function to add layer name, maximum zoom level, minimum zoom level to each source file and output them to JSON sequence to be converted into vector tile. There is a "document" function that shows the layer name, attribute name, and zoom level range from the vector tile metadata. By passing the filtered data to Tippecanoe, you can easily create a vector tile with adjusted layer structure. (I'm a Windows user, so I often use this tool with unvt / nanban for some purposes.)

image.png
(The figure is from the GitHub repository. Although it is an initial concept, you can understand that ① is the document function and ② is the filter function.)

  • unvt/kata can easily give the layer name and maximum / minimum zoom level for each source by using the filter function when creating vector tiles. You can easily create a vector tile by passing the filtered one to tippecanoe with the following command.
kata filter kata.yml | tippecanoe -e zxy2 --no-tile-compression --no-feature-limit --no-tile-size-limit
  • I think it is a very useful tool because you can check the layer list and property list of vector tiles.
  • If the data exceeds several GB, there is no setting to divide the output file according to the region, so I think that something other than this tool is needed, but it perfectly fits for small data conversion.

Repository URL:

Past efforts and references:

unvt/itoma

overview:

It displays vector tile map styles (YAML format imported by charites) in multiple vector tile libraries (Mapbox GL JS, MapLibre GL JS, ArcGIS API for Javascript) ) as preview by localhost. In addition, since it returns a response based on the VectorTileServer service of the ArcGIS REST API, you can check the use of vector tile maps in ArcGIS Online. (Note: Localhost needs to be https) (I'm a Windows user, so I often use this tool with unvt / nanban for some purposes.)

image.png

Repository URL:

Past efforts and references:

unvt/nanban

overview:

Ubuntu/Intel based UNVT docker file. Even Windows users can get a Linux-like working environment through the Docker container, so vector tile conversion using tippecanoe can be done easily. Since gdal (ogr2ogr), nodejs, ruby, etc. are also included, it can be used not only for vector tile creation but also for hosting test and style test.

The main components of this tool are:

  • nodejs
  • gdal-bin
  • ruby
  • sqlite3
  • tmux
  • vim
  • tarn
  • mapbox/tippecanoe
  • budo
  • hjson
  • pm2
  • phshcorn/hocon-parser
  • vt-optimizer (It doesn't work with recent nodejs, so I haven't put it in the current nanban.)
  • unvt/charites
  • unvt/kata
  • unvt/itoma

Please read carefully as the policy regarding the use of Docker changed in the summer of 2021 ([Docker FAQ] (https://www.docker.com/pricing/faq/)).

Docker Desktop may be used for free as part of a Docker Personal subscription for:
..
Non-commercial open source projects

Repository URL:

Past efforts and references:

unvt/equinox

overview:

UNVT related tool installer for Raspberry Pi. You can install tools related to vector tile processing by curling and running the installer from Raspberry Pi.

As for the component of the tool, I will not write it in detail here because it is easy to understand by looking at @hfu's article. In addition to the general tools such as nginx, nodejs and ruby, it also contains geospatial and vector tile related tools like Tippecanoe, gdal, pdal, charites, vt-optimizer.

A few years ago, we often run a Docker container on a Raspberry Pi, but I think that by using this equinox, it is possible to build a stable environment without using a Docker image.

Repository URL:

Past efforts and references:

Acknowledgments

From the beginning to the present, @hfu's contributio is wonderful. Thank you for your daily efforts. We have also been happy to have a large number of partners as the UNVT community. In particular, I think the fact that Georgia donated unvt/charites is a big step forward. Thank you.
We would like to thank all the university students for participating in the UNVT hackathon in December 2021. I am also looking forward to seeing what progress will be made in the future.

References

(For reference of each tool, citations and links are summarized in the text.)

1
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?