RU | EN

MFW Export Scripts

A pipeline for preparing offline data (tiles, fonts, styles, routing and geocoding data) for displaying a map in the MFW Example.

Table of Contents

• Terms of Use and Participation
• Project Structure
• Compatibility
• Usage
  • 1. Uploading files
  • 2. Uploading the necessary data to the device
    • 2.1. Uploading tiles (mbtiles type)
    • 2.2. Uploading tiles (mvtiles type)
    • 2.3. Uploading fonts
    • 2.4. Uploading styles
    • 2.5. Uploading routing data
    • 2.6. Uploading geocoding data
  • 3. Launching the application
  • 4. script_data usage scenario
  • 5. Map Export Examples
    • 5.1 Exporting St. Petersburg
    • 5.2 Exporting the Russian Federation

Terms of Use and Participation

The source code of the project is provided under the license, which allows its use in third-party applications.

The contributor agreement documents the rights granted by contributors of the Open Mobile Platform.

Information about the contributors is specified in the AUTHORS file.

Code of conduct is a current set of rules of the Open Mobile Platform which informs you how we expect the members of the community will interact while contributing and communicating.

Project Structure

• MfwExportScripts directory contains the source code of the project.
• examples directory contains already generated fonts and a file with tiles of mbtiles type for St. Petersburg as an example.
• output directory contains the result of the sh script usage.
• screenshots directory contains a screenshot with an example of the used directories.

Compatibility

The project is compatible with all the supported versions of the Aurora OS.

Usage

1. Uploading files

• Clone the repository MfwExportScripts.
• Download the *.osm.pbf file (in the following, the OSM file) with the required region from the Geofabrik website. (Russian Federation maps).
• Use the OpenStreetMap website and find the desired city/region by its name.
• Save the city/region code from the header of the selected city/region. Example: Relation: Saint Petersburg (421007), where 4271007 - is the city code.
• Run the run.sh script from the repository. Be aware that the script requires Docker to operate. Script arguments:

  sh run.sh <OSM file name> <region code> <tile type (mbtiles or mvtiles)> \ 
  <absolute path to the mounted volume (for example: /Users/SomeUser/Documents/osmpbf_data)>

• At the first run the run.sh script will build an image, install dependencies and launch the container. Subsequent runs are also carried out through the run.sh script with the specified parameters. For the script to work correctly with the OSM file, the file must be located in a user-mounted directory.
• The mounted directory must be located in one of the directories (the example uses the Users/ directory):
• After the work is completed, the directory output will contain:
  • A file with output.mbtiles tiles or the directory mvtiles/ (depending on the specified argument);
  • Archive with fonts glyphs.tar.bz2;
  • Directories global-v1/../ and country-v1/../ with data for libpostal;
  • Tar-archive with routing tiles valhalla_tiles.tar;
  • The directory geocoder-nlp/, containing (geonlp-primary.sqlite, geonlp-normalized.trie, geonlp-normalized-id.tkh) data for the geocoder.

2. Uploading the necessary data to the device

2.1. Uploading tiles (mbtiles type)

• Connect the device in data transfer mode.
• Copy output.mbtiles and glyphs.tar.bz2 to the device, for example, to the Documents directory.

2.2. Uploading tiles (mvtiles type)

• Connect the device in data transfer mode.
• Copy the directory mvtiles/ and glyphs.tar.bz2 to the device, for example, to the Documents directory.

2.3. Uploading fonts

• Unzip fonts using the command: mkdir glyphs tar -xf glyphs.tar.bz2 -C glyphs
• When working with mvtiles tiles, the font directory must be placed inside the tile folder.

2.4. Uploading styles

• Clone the MfwExample repository.
• Set all dependencies with the command: git submodule update --init --recursive
• In the MfwExample files map_light_theme.json for the light map theme and dark_matter.json for the dark map theme exist.
  • mbtiles type. The theme files must be moved to the same directory where output.mbtiles and fonts are located. In this case, this is the Documents folder (not necessarily because the application will have a file picker). The correct paths for "url" and "glyphs" must be set in the style files. Styles should have the following lines:

    "sources": {
        "openmaptiles": {
          "type": "vector",
          "url": "mbtiles://{TILES_PATH}/output.mbtiles"
        }
      },
    "glyphs": "file://{TILES_PATH}/glyphs/{fontstack}/{range}.pbf",
    "layers": [
    {
      "id": "background",
      "type": "background",
      "paint": {"background-color": "#162639"}
    }

  • mvtiles type. Files with themes must be moved inside the folder with tiles - "mvtiles". Styles should have the following lines:

    "sources": {
        "openmaptiles": {
            "type": "vector",
            "tiles": ["file://{TILES_PATH}/{z}/{x}/{y}.pbf"],
            "maxzoom": 15
        }
    },
    "glyphs": "file://{TILES_PATH}/glyphs/{fontstack}/{range}.pbf",
    "layers": [
    {
        "id": "background",
        "type": "background",
        "paint": {"background-color": "#162639"}
    }

2.5. Uploading routing data

• Connect the device in data transfer mode.
• Copy valhalla.json valhalla_tiles.tar to the device, for example, to the Documents directory.
• The generated data for routing works only for the region/city specified during data generation. (relation from the OpenStreetMap website).

2.6. Uploading geocoding data

• Connect the device in data transfer mode.
• Copy global-v1/, geocoder-nlp/, country-v1 to the device, for example, to the Documents directory.
• The generated geocoding data works only for the region/city specified during data generation (relation from the OpenStreetMap website).

3. Launching the application

To verify the exported data, you can use the MFW Example application. MapTemplate does not have an MBTiles demonstration.

• mbtiles

  • Build MFW Example application through the IDE, connect the device and install the application.
  • After launching the application, select MBTiles.
  • Press 'Select a file with styles' and select one of the style files previously downloaded to the device.
  • If everything is done correctly, an offline map of the desired region with the selected style should be displayed.

• Routing

  • In the application, go to the "Routes" tab and click 'Route between two points'.
  • Press 'Select valhalla configuration file' and specify the configuration file.
  • A route building interface will appear on the map with selection of start and end points.

• Geocoding

  • In the application, go to the "Geocoding" tab and click "Auto-address".
  • Press 'Select the path to postal/global' and specify the path to folders global-v1, country-v1, geocoder-nlp.
  • Click the 'Search' button and enter the required address.

4. script_data usage scenario

If there is a problem with downloading planetiler static files, you must:

• Download lake_centerlines.shp.zip.
• Download water-polygons-split-3857.zip.
• Download natural_earth_vector.sqlite.zip.
• Copy the downloaded files to the container in which the script was executed. Example of a copy command:

  docker cp /<absolute path to the downloaded file>/lake_centraline.shp.zip \ 
  mfwexportscripts:/app/mapbox/planetiler/data/sources/
  docker cp /<absolute path to the downloaded file>/water-polygons-split-3857.zip \ 
  mfwexportscripts:/app/mapbox/planetiler/data/sources/
  docker cp /<absolute path to the downloaded file>/natural_earth_vector.sqlite.zip \ 
  mfwexportscripts:/app/mapbox/planetiler/data/sources/

• Run the script again using the command:

  sh run.sh <OSM file name> <region code> <tile type (mbtiles or mvtiles)> \ 
  <absolute path to the mounted volume (for example: /Users/SomeUser/Documents/osmpbf_data)>

5. Map Export Examples

5.1 Exporting St. Petersburg

• Create a directory where we will download the *.osm.pbf file. It should be accessible for creating a docker volume. In this example, the directory /var/spb/ will be created.
• Download data for St. Petersburg and place it in the /var/spb/ directory.
• Run the script with the city code:

sh run.sh northwestern-fed-district-latest.osm.pbf 421007 mbtiles /var/spb/

You can optionally replace mbtiles with mvtiles

5.2 Exporting the Russian Federation

• Create a directory where we will download the *.osm.pbf file. It should be accessible for creating a docker volume. In this example, the directory /var/russia/ will be created.

• Download data for Russia and place it in the /var/russia/ directory.

• Increase the amount of RAM allocated to java in mbtiles_extract.sh to at least 16 gigabytes:

...
java -Xmx16g
...

Run the script with the code of Russian Federation:

sh run.sh russia-latest.osm.pbf 60189 mbtiles /var/russia/

Minimum hardware requirements for running:

• 32 GB RAM
• processor with 8 threads
• 350 GB disk space

You can optionally replace mbtiles with mvtiles

This document in Russian / Перевод этого документа на русский язык

• README.ru.md