Web maps: creating self-hosted and self-styled vector maps for your website
- Change map style:
TL;DR: if you are not interested in the background you can jump right to the "Creating vector tiles and using JSON data to style the map" section.
I like maps.
Maps are often beautiful and tell stories. I like (and collect) printed maps but can also spend hours diving into digital maps and satellite images. Maps are sometimes also part of my work as a web developer. Most of the time it's about embedding Google Maps on the contact page. Sometimes even themed. I think it's a pity that maps mostly look the same on more or less every website. I think it's time to change that. Also due to privacy reasons.
How maps are usually used on the web
In my experience as a web developer most of the time Google Maps will be used on websites to show geographical information. Often it's about showing the location of a business/institution/... with a pin as well as the surroundings. It's either done via an iframe or the Google Maps API. Especially using the iframe is very simple. The "problem" with using Google Maps (from the perspective of a developer located in the EU) is though, that data is loaded from Google's servers and therefore personal data of the website visitor gets transferred to an "insecure third country". The personal data I'm referring to here is the IP address which is used for every connection. You need the consent of the user before loading any data from Google (or any other US based companies). There are ways to do this but you still should have a fallback for the case, that the consent is not given (yet).
Another approach is using OpenStreetMap (OSM) data in combination with a library to implement it on the website. I'm talking about libraries like Leaflet or OpenLayers.
When using OpenStreetMap data you can choose where to load it from and opt for a server in the EU to not face the data protection issues mentioned before.
What I don't like about the maps you get from OSM though is the default style. There are different styles you can choose from when visiting the link above. But not all of the services are located in the EU ...
Often, my goal is to create a website which is independent of any external resources. No matter if inside the EU or outside. So I self-host scripts, videos, fonts and so on. But with maps, which are sometimes needed in my projects, I didn't have a solution to self-host the map tiles. So I started doing research about it and want to share my findings here.
What are map tiles
When using interactive maps on the WWW those maps are often cut into pieces and served as multiple images instead of one large image. This brings performance advantages since you often only look at a small clipping of the whole map. The libraries mentioned above have algorithms implemented to calculate which tiles are needed for the currently shown part of the map and request those from the tile server and show them.
Zoom levels and generalisation
Interactive maps are usually zoomable. When zooming in or out of a map, the elements in the map may change. Some appear, some dissapear. In cartography this is called generalization. When viewing a map of Europe, only the biggest cities and country names as well as country borders and big bodies of water may be visible. Showing labels and points for every city on a large scale would make it impossible to "read" the map. But that's what's so great about interactive maps. You can zoom in to see certain areas in more detail and get more information about it. At a certain zoom level not only cities but also villages will be shown. Zooming further in even street names and buildings will appear.
The map tiles need to be generated for every zoom level seperately also because the contents differ on the different zoom levels. When zooming completly out of the map, there may be only some tiles needed to cover the whole area. On smaller zoom levels some hundreds or thousands may be needed because every tile covers a much smaller area.
Approach 1: Creating your own raster tiles from OpenStreetMap data
On the Switch2OSM Website I found a great tutorial on how to create and serve map tiles yourself. You can find it here. The tutorials for serving tiles are available for Ubuntu machines. Since I use a Mac with Apple Silicon I chose to use Multipass to create a virutal machine to be able to try it out. Following the tutorial was straight forward. After allocating enough resources to Multipass I had my own tile server up and running. When following the Leaflet Quick Start Guide one only needs to change the URL for the tileLayer to have the interactive web map working on a website.
Styling the map
The next thing I wanted to do is to change the style of the map. Following the tutorial the OSM default styles were loaded from "OpenStreetMap Carto" and converted into a .xml file for Mapnik where the styles are defined. When talking about styling a map it's about setting colours for different elements (streets, surfaces, buildings, borders, ...), fonts for labels and also which elements are shown on which zoom levels to name just a few things.
Looking at the project.mml file or the generated mapnik.xml file, I found it quite overwhelming to adapt those style definitions to something I find more appealing. The elements in the OSM data have many different categories and especially for someone rather new to OSM data I found it confusing.
I thought it would be good to find a different stylesheet (.mml or .xml file) I can use with Mapnik but couldn't find a showcase or something where multiple styles are presented and one can download them. Also I wasn't so happy about the resolution of the raster tiles (I guess that's something I would be able to change) but would prefer vector tiles anyway.
The other reason I was looking for alternatives is that I usually use shared hosting for my projects. So it's not possible to run the "OSM tile server stack" there because I can't install anything on OS level. I thought about running it locally, writing a script that creates all the tiles and just uploading the image tiles to the webserver. But this wouldn't be an ideal solution.
Approach 2: Creating vector tiles and using JSON data to style the map
When doing research about web maps I came across services like mapbox.com, maptiler.com or openmaptiles.org. They provide great maps as well as special GIS services. Using one of those wasn't an option because it would require loading external data. But on the OpenMapTiles website I found a tutorial for serving vector tiles with php. Since I'm usually working with a LAMP stack I had a look at it and found it very helpful. I needed to make some adaptions to fit my needs, here is how I did it.
Step 1: Downloading TileServer PHP
TileServer PHP is a free and open-source project you can find on Github. All you need to do is to download the latest release and upload it to your server.
Step 2: Providing vector tiles
For TileServer PHP to be able to serve vector tiles you need to provide those data. At this point you need to decide which area you want to cover. Is it necessary to have the whole world covered or is it enough to have a city, region or country covered? Depending on your decision there are different sources you can get the data from.
- Geofabrik provides OpenStreetMap data to download for all regions. You can download whole continents, countries or sub regions of countries there for free.
- Protomaps provides a tool to download a custom area of OSM data. Here you can draw a rectangle or polygon and download this area. May be useful if you only need a city or some area which lies in multiple countries.
Step 3: Converting the vector file
The OSM vector data you get has the extension .osm.pbf. For TileServer PHP to handle the data, it needs to be in the .mbtiles format. I used a tool called tilemaker for this. You can then download the latest release for your platform.
After downloading it you need to run a simple command on the command line:
tilemaker --input inputfile.osm.pbf --output outputfile.mbtiles --process resources/process-openmaptiles.lua --config resources/config-openmaptiles.jsonFor the first run, I recommend using the provided config and process files. (I will come back to this in the next step.) You may need to change the path to the input file and the path to the process and config files.
Depending on the size of the area, the details and your machine, this may take some minutes to finish.
Afterwards copy the outputted .mbtiles file in the TileServer PHP directory on your server.
When calling the URL of the folder you uploaded TileServer PHP to, you should see a preview of the vector data. It's kind of unstyled but that's ok for now. You will find a link labled "TileJSON" there. When clicking it, you get redirected to a JSON file. Copy the URL.
Step 4: Defining the styles
One reason I stopped following approach 1 was that I found it difficult to style the map. A tool like Maputnik on the other hand provides great presets (click on Open) and lets you play around with the settings while giving you a preview of it immediately. You can select which features should be included, how they should look like and at what zoom levels they should appear. If you want to find out more about how to style the map, have a look at the Maputnik Wiki. If you want to find out which features are included in the OSM data downloaded from Geofabrik (and therefore can be styled), have a look into the documentation. Be aware that many features get removed when generating the .mbtiles files with tilemaker though. Check the .json file used with tilemaker to see what will be included.
To have the needed layers really included (and maybe only those), you may need to change the .json config file for tilemaker. When having a look at it you find the layers which will be included as well as minzoom and maxzoom attributes for them. As soon as you know which layers you want to show at which zoom levels, you need to adjust the values in the maptiler config file. You can also just set the minzoom to 0 and maxzoom to 16 (a higher value didn't work for me. I think the OSM data from Geofabrik limits it) but then the process of generating the .mbtiles file takes much longer and the resulting file is much larger.
When you are done with the styling in Maputnik, click export and download the .json file. Next open the .json file in a text editor. You will find a "sources" attribute in there, which may look like this:
"sources": {
"openmaptiles": {
"type": "vector",
"url": "https://api.maptiler.com/tiles/v3/tiles.json?key={key}"
}
},Change the URL to the one you copied at the end of the previous step (TileJSON from TileServer PHP).
You may find other URLs to external resources in the .json file containing the style definitions. If you don't want to use any external resources, you should remove those attributes. For the fonts to be rendered correctly on the map, you need to serve them as web fonts in your CSS. (I also wrote an article about self hosting Google Fonts in case you are interested.)
Step 5: Showing vector tiles on your website
I'm a fan of Leaflet because it's simple and lightweight. Unfortunately I couldn't manage to get Leaflet working with vector tiles. So I had to switch to OpenLayers. As a starting point, I used the code from the Quick Start tutorial.
Next I needed to download the ol-mapbox-style library to use the created styles with the vector data. Since I don't want to use npm, I downloaded the latest release and copied the file olms.js from the dist folder. Create a script element in your html file and link the olms.js.
Next, I removed the layer property from the sample code because I don't want to use the OSM data.
<body>
<div id="map" style="aspect-ratio: 16 / 9;"></div>
<script src="ol.js"></script>
<script src="olms.js"></script>
<script>
const map = new ol.Map({
target: "map",
view: new ol.View({
center: ol.proj.fromLonLat([-19.6, 64.7]),
zoom: 7,
})
});
olms.apply(map, "map-styles.json");
</script>
</body>When you open your HTML file in a browser, you should now see your vector data with the styles applied. If you can't see anything, you might have to change the values of the view attribute. Set the zoom to 1 to fully zoom out or set the center attribute to coordinates inside the area you provided tiles for. Another problem that could occur is connected to CORS. You should see hints about it in the developer console. To fix this you need to set the "Access-Control-Allow-Origin" in the .htaccess file of TileServer PHP. The code is already inside the file, you just need to remove the "#".
Step 6: Removing external webfonts
When using the ol-mapbox-style library external fonts will be loaded from Google Fonts. Open the ol-mapbox-style.js file in a text editor, search for "google" and just replace the URL with an empty string. Further, remove the part with "document.head.appendChild" which should be closeby (but hard to find in the minified version).
Drawbacks of this method
- One drawback of creating and serving vector tiles this way is that they won't be updated when changes in the OSM data are made. Depending on your needs regarding "up-to-dateness" you need to create a new .mbtiles file and replace it on the server from time to time.
Further resources
Additional to the resources I have linked in the text, here some websites I found useful during my research: