lidar2map — free tool to generate LiDAR relief maps (hillshade / SVF) for Locus

[nico579]

Hi Col,

Thanks for confirming, glad the Gatekeeper workaround got you running, and that the tile border is gone now that the map is used as an overlay.

While testing multi-block splits myself, I found and fixed a related class of bug that might be relevant to what you're seeing when zoomed out on a split map. Each block's output extent was computed independently, and a straight edge does not stay perfectly straight once you reproject from the source CRS to Web Mercator. Two separate issues came from that: neighboring blocks disagreed slightly on where their shared edge actually was, and at points where four blocks meet, each of the four computed a slightly different position for that shared corner, leaving a small area with no published tiles right at the junction.

Both are now fixed. Adjacent blocks compute their shared boundary from the exact same reference points, so they agree exactly. For the four-way corner, each block now merges in the real, already-computed data from its neighbors before warping, so the small gap gets filled with real pixels rather than being left empty. This will be in the next release.

That said, I'm not yet sure this is the same bug behind your grey box. Your symptom is specifically when a map is used as the main map and you zoom out past the tile border, not at an internal seam between two blocks, so it could be a distinct issue, possibly how Locus handles areas outside a layer's coverage when it's the base map rather than an overlay. Could you help me narrow it down:

- Was the file you tested a single (non-split) MBTiles, or did it come from a multi-block split?
- MBTiles or RMAP? (you mentioned it doesn't happen with your sqlitedb maps, so I assume MBTiles)
- A screenshot of the grey box would help a lot.
- Does it appear right at the edge of the map's actual coverage, or only once you're zoomed out past its max data zoom?

Thanks again, your testing has been extremely valuable, especially with the Intel Mac access.

[Col]

Hi.
It is a non split mbtiles map, and the grey box is there no matter the zoom, but it gets proportionally bigger compared to the map tile coverage as you zoom out. It also disappears when you get to zoom level 8, and it disappears on 2 sides at zoom 18 (top and bottom) with one side grey and the other a black area.

https://photos.app.goo.gl/eUwJKJkAvjD6S2e66

https://photos.app.goo.gl/6bFmoyczov1nn7Dv9

[nico579]

Hi Col,

Thanks, this description is very helpful. I was not able to open the photo links on my end, so what follows is based on your written description rather
than the screenshots themselves, let me know if I got any detail wrong.

I think these are actually two different things.

The black one is a real bug I found and fixed.
When lidar2map reprojects the source raster from its native CRS (Lambert93 for France) to Web Mercator
for tiling, a rectangle in the native CRS does not stay perfectly rectangular after reprojection,
it comes out very slightly rotated.

The warped file is still written as an axis-aligned rectangle (the bounding box of that rotated shape),
so its own corners contain a thin wedge that falls outside the true data footprint.
The reprojection step was not marking that wedge as nodata, so it got filled with black and treated as if it were real data.

On the test zone I used to reproduce this, about 2% of the pixels near the corners were affected.
This is now fixed: the warp step also produces a proper coverage mask, and the tiler uses it to make those wedge pixels transparent instead of opaque black.
It will be in the next release, and it applies to non-split maps too, so it should be exactly your case.

The grey one I believe is not a lidar2map bug, but how Locus itself handles a layer with limited coverage when it is used as the main map rather than an overlay.
Outside the area lidar2map actually covers, there is no tile to request at all, and Locus has to render something for that space, likely its own neutral background.
That would also explain why it disappears at very low zoom (probably below the map's declared minzoom, where Locus stops requesting the layer altogether)
and why it behaves differently from the black artifact, which is a real opaque pixel inside a real tile, not a gap.

If you get a chance to test the next build once it is out, I would be curious whether the black area is gone and whether the grey one is still there
and behaves the same regardless of shading type. That would confirm the two are unrelated.

Thanks again for the detailed writeup, and sorry I could not view the screenshots directly, if I misread anything from the description just let me know.

[Col]

Thanks for your detailed explanation. Sorry about the photos, i accidently deleted them but the links should work again now. I'll test once the release is out.

[nico579]

Thanks, the links worked this time.

Both screenshots match what I expected:

The black band one is exactly the nodata wedge bug, confirmed. It's already fixed and live in the current build, you don't need to wait for a separate release announcement, a fresh download/update should have it now.

The grey checkerboard right next to the black band is not a lidar2map issue, it's Locus's own placeholder for "no tile available here." That one is expected behavior, not something to fix.

The zoomed out overview (whole trek, 20 km scale) also makes sense: I can see a couple of rectangles with sharp edges (one around the 462 m point, another one further up with several of your day checkpoints). Those are just the boundaries of the zones you generated with lidar2map, the base map fills in outside them. Same idea as the grey checkerboard, just at a bigger scale.

So after you update: the black band should be gone, but you'll still see the sharp rectangle edges and occasional grey checkerboard at the boundary of your covered zones, that part is normal and not going to change.

One tip that should help with the grey checkerboard specifically: set the lidar2map shading as your main map and OSM as an overlay on top, rather than the other way round. OSM is vector, so it always renders everywhere with no gaps, whereas the shading only exists inside the rectangle you generated. With OSM layered on top, it fills in the checkerboard outside that rectangle instead of leaving it blank, while the relief shading stays the dominant thing you see inside the covered area.

For that OSM overlay, you can generate it with lidar2map itself, no need for a separate tool:
--osm --layer "highway=path highway=track highway=footway" --file-formats map
gives you just the paths/tracks as a Mapsforge .map file, which is what Locus wants for an overlay.

On the speed question: numba is included in all the builds (Windows/Mac/Linux), so it should already be active on your end.
You'd see a line like "SVF Numba JIT: compiling on first call" near the start of a run if it's kicking in.
If it's still slow for large areas, that's mostly a hardware/scale thing rather than something the build is missing, SVF and openness over a big zone are genuinely CPU heavy.
If you want it faster on big areas, running it on a cloud VM (a few euros/hour on something like Hetzner) via lidar2map's `--remote-cli` mode is an option, it just gives you more cores than a laptop for the duration of the run.

[Col]

Hi Nico,
all has been working great apart from one thing a found.Im using bounding boxes and have an area of the Lake district selected for which the corner juts out into the sea near where there is a nuclear submarine manufacturing facility. There is an area of sea which im assuming belongs to that, but rather than a 'no coverage' being returned, i get lots of Http 500 errors then the process fails. if i reduce the box a little to exclude the area -3.421000, 54.138400, -3.376200, 54.210300 then all is good :

Code Select Expand

$ /Users/mus/Library/Application Support/lidar2map/lidar2map --provider gb-england --zone-bbox -3.512468,54.144741,-2.624122,54.959754 --zone-width 20 --zone-name Lakes --output-dir /Volumes/Reservoir-A/_Colonel/Maps/_Lidar2map/Lakes/ --cache-dir /Volumes/Reservoir-A/_Colonel/Maps/_Lidar2map/-cache/ --production-dir /Volumes/Reservoir-A/_Colonel/Maps/_Lidar2map/-production/ --lidar --download --workers 8 --shading vat:dist=20,gamma=2 --shading opos:dist=20,gamma=2,sweep=1 --svf-sweep --file-formats mbtiles --zoom-min 8 --zoom-max 17 --image-format jpeg --image-quality 85

  Log : /Users/mus/Applications/logs/lidar_20260821_101849_20630.log (publication atomique à la fin)
  History: entry '1787303928287-11242-gui' (in progress)
=======================================================
  LiDAR : Royaume-Uni (Angleterre) — LIDAR Composite DTM (EA)
  Pipeline rasterio + numpy (numba for SVF)
=======================================================
  Folder : /Volumes/Reservoir-A/_Colonel/Maps/_Lidar2map/Lakes/
  BBox WGS84 : -3.5125,54.1447 → -2.6241,54.9598
  BBox EPSG:27700 : 301301,472281 → 360133,563838
  Area: ~5386 km²
STEP:1/3 Downloading tiles
  -> Compression enabled
  Root    : /Volumes/Reservoir-A/_Colonel/Maps/_Lidar2map/Lakes
  Tiles  : /Volumes/Reservoir-A/_Colonel/Maps/_Lidar2map/-cache/lidar/gb
  Zone    : /Volumes/Reservoir-A/_Colonel/Maps/_Lidar2map/Lakes
  LiDAR tiles [░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  0%  24/5391  19s
  ERROR dtm_england_301_489.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  2%  115/5391  1m37s
  ERROR dtm_england_302_489.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  3%  206/5391  2m37s
  ERROR dtm_england_303_489.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  5%  283/5391  3m34s
  ERROR dtm_england_304_479.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  5%  284/5391  3m39s
  ERROR dtm_england_304_481.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  5%  285/5391  3m39s
  ERROR dtm_england_304_483.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  5%  286/5391  3m40s
  ERROR dtm_england_304_482.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  5%  287/5391  3m40s
  ERROR dtm_england_304_480.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  5%  289/5391  3m41s
  ERROR dtm_england_304_484.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  5%  290/5391  3m43s
  ERROR dtm_england_304_485.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  5%  291/5391  3m44s
  ERROR dtm_england_304_486.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  5%  295/5391  3m49s
  ERROR dtm_england_304_487.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  5%  296/5391  3m50s
  ERROR dtm_england_304_488.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [██░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  7%  379/5391  4m57s
  ERROR dtm_england_305_479.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [██░░░░░░░░░░░░░░░░░░░░░░░░░░░░]  8%  480/5391  6m11s
  ERROR dtm_england_306_479.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [███░░░░░░░░░░░░░░░░░░░░░░░░░░░]  10%  569/5391  7m19s
  ERROR dtm_england_307_479.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [███░░░░░░░░░░░░░░░░░░░░░░░░░░░]  12%  656/5391  8m28s
  ERROR dtm_england_308_479.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [████░░░░░░░░░░░░░░░░░░░░░░░░░░]  13%  736/5391  9m23s
  ERROR dtm_england_309_472.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [████░░░░░░░░░░░░░░░░░░░░░░░░░░]  13%  737/5391  9m27s
  ERROR dtm_england_309_473.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [████░░░░░░░░░░░░░░░░░░░░░░░░░░]  13%  738/5391  9m27s
  ERROR dtm_england_309_474.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [████░░░░░░░░░░░░░░░░░░░░░░░░░░]  13%  740/5391  9m29s
  ERROR dtm_england_309_475.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [████░░░░░░░░░░░░░░░░░░░░░░░░░░]  13%  745/5391  9m32s
  ERROR dtm_england_309_476.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [████░░░░░░░░░░░░░░░░░░░░░░░░░░]  13%  748/5391  9m33s
  ERROR dtm_england_309_477.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [████░░░░░░░░░░░░░░░░░░░░░░░░░░]  13%  750/5391  9m34s
  ERROR dtm_england_309_478.tif (OSError, attempt 3): HTTP 500
  LiDAR tiles [███████░░░░░░░░░░░░░░░░░░░░░░░]  23%  1279/5391  17m24s
  ERROR dtm_england_314_511.tif (TimeoutError, attempt 3): The read operation timed out
  LiDAR tiles [███████░░░░░░░░░░░░░░░░░░░░░░░]  26%  1410/5391  21m39s
  ERROR dtm_england_316_483.tif (TimeoutError, attempt 3): The read operation timed out
  LiDAR tiles [██████████████████░░░░░░░░░░░░]  61%  3320/5391  46m45s
  ERROR dtm_england_337_482.tif (TimeoutError, attempt 3): The read operation timed out
  LiDAR tiles [██████████████████░░░░░░░░░░░░]  62%  3356/5391  48m56s
  ERROR dtm_england_337_524.tif (TimeoutError, attempt 3): The read operation timed out
  LiDAR tiles [██████████████████░░░░░░░░░░░░]  62%  3358/5391  49m31s
  ERROR dtm_england_337_529.tif (TimeoutError, attempt 3): The read operation timed out
  LiDAR tiles [██████████████████████████████] 100%  5391/5391  1h12m47s
  Downloaded: 5362  Cache: 129  Missing: 0  Errors: 29
  History saved: /Users/mus/Applications/historique.json  (9 entries)
[PYI-20630:ERROR] Failed to execute script '_loader' due to unhandled exception: 29 tile download error(s) - pipeline stopped before shading/tiling (rerun to retry the failed tiles; successful ones are cached)
[PYI-20630:ERROR] Traceback:
Traceback (most recent call last):
  File "_loader.py", line 36, in <module>
  File "<frozen runpy>", line 287, in run_path
  File "<frozen runpy>", line 98, in _run_module_code
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/mus/Library/Application Support/lidar2map/_internal/lidar2map.py", line 11116, in <module>
    _dispatch()
  File "/Users/mus/Library/Application Support/lidar2map/_internal/lidar2map.py", line 11031, in _dispatch
    return main()    # --lidar / --osm / par défaut
          ^^^^^^
  File "/Users/mus/Library/Application Support/lidar2map/_internal/lidar2map.py", line 5564, in main
    _telecharger_dalles_zone(dalles_dict, bbox, dossier_dalles, dossier_ville, args)
  File "/Users/mus/Library/Application Support/lidar2map/_internal/lidar2map.py", line 6156, in _telecharger_dalles_zone
    return _telecharger_dalles_zone_impl(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "_terrain_download.py", line 255, in telecharger_dalles_zone
RuntimeError: 29 tile download error(s) - pipeline stopped before shading/tiling (rerun to retry the failed tiles; successful ones are cached)
UNHANDLED EXCEPTION:
Traceback (most recent call last):
  File "_loader.py", line 36, in <module>
  File "<frozen runpy>", line 287, in run_path
  File "<frozen runpy>", line 98, in _run_module_code
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/mus/Library/Application Support/lidar2map/_internal/lidar2map.py", line 11116, in <module>
    _dispatch()
  File "/Users/mus/Library/Application Support/lidar2map/_internal/lidar2map.py", line 11031, in _dispatch
    return main()    # --lidar / --osm / par défaut
          ^^^^^^
  File "/Users/mus/Library/Application Support/lidar2map/_internal/lidar2map.py", line 5564, in main
    _telecharger_dalles_zone(dalles_dict, bbox, dossier_dalles, dossier_ville, args)
  File "/Users/mus/Library/Application Support/lidar2map/_internal/lidar2map.py", line 6156, in _telecharger_dalles_zone
    return _telecharger_dalles_zone_impl(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "_terrain_download.py", line 255, in telecharger_dalles_zone
RuntimeError: 29 tile download error(s) - pipeline stopped before shading/tiling (rerun to retry the failed tiles; successful ones are cached)

✗ Terminé (code 1)
  History saved: /Users/mus/Applications/historique.json


[nico579]

Hi Col,

Thanks for the detailed report, that pinpointed it fast.
Confirmed and fixed: your bbox corner falls outside the real EA LIDAR coverage (sea near the Barrow-in-Furness naval yard), and their WCS server returns HTTP 500 for those out-of-coverage tiles instead of a clean 404. lidar2map was treating 500 as a hard error, so after 3 retries per tile it aborted the whole run, even though 5362/5391 tiles (99.5%) had already downloaded fine.

Fix: gb-england now treats that specific WCS 500 the same as a missing tile (skip, don't fail).
It's in v1.47.1, just tagged, build is running now (~30 min for all 3 platforms), should be up within the hour.

Your workaround (trimming the bbox) still works fine in the meantime, but you shouldn't need it once you update.

[Col]

thanks for your quick fixes. I did mean to ask what if anything you are using to merge .mbtiles files?

[nico579]

Update:
I extended --merge to handle .mbtiles sources directly now, not just GeoJSON.
Same flag, same GUI tab (now just called "Merge"), it detects the file type from the sources you give it:

python lidar2map.py --merge --source zone_nord.mbtiles zone_sud.mbtiles --output-file merged.mbtiles

Bounds and zoom range become the union of the sources;
if two sources overlap on the same tile, the last one listed wins.
Add --file-formats rmap sqlitedb if you also want those outputs.
This will land in the next release.

[Col]

Again great stuff. one more error i occasionally notice is:

Code Select Expand

✓ Step 3 finished in 44m25s  (cumulative 2h13m40s)
  (index sheet: reverse geocoding failed: URLError: <urlopen error [Errno 8] nodename nor servname provided, or not known>)
  lakes_opos_20m_g2p0_ombrage_z8-17_planche.png : index sheet (1000x968)
  lakes_vat_20m_g2p0_ombrage_z8-17_planche.png : index sheet (1000x968)
  Done! Folder: /Volumes/Reservoir-A/_Colonel/Maps/_Lidar2map/North Pennines
  Total time: 133m40s
  History saved: /Users/mus/Applications/historique.json  (20 entries)

✓ Terminé (code 0)
  History saved: /Users/mus/Applications/historique.json

[nico579]

That error is a DNS lookup failure (errno 8 = getaddrinfo couldn't resolve the hostname),
not something that breaks your run: it's from a best-effort reverse-geocoding call (looking up which department your zone is in, purely to draw its outline on the index sheet).
It was already non-fatal by design, which is why your run finished fine with code 0 and both index sheets got produced.

The "occasionally" was a transient network blip at that exact moment on a long run.
I added a retry (one retry, 2s wait) around that specific call in v1.48.2, currently building.
Should cut down how often you see it.