Messages

In the end I was able to gain access to my GEMF maps by moving them with adb.  The trick was to place them in the Android/media tree on the SD card, which does permit write access.  I'm not terribly happy with the resulting file system hierarchy, with maps scattered around three different directories, but it works.

Well, it mostly works.  Importing my Lomaps one by one, the ended up in the same directory, and Locus thinks they are all part of Europe, whereas they are actually a mixture of European country maps and US/Canada state/province maps.  At least they are all accessible.

I recently updated my phone to Android 11 (LineageOS 18.1), very innocently assuming that I could just copy my existing raster/vector map directories to someplace where the new Locus Classic install would find them.

Close to a full day later, I managed to transfer everything off of the SD card using adb, so it could be formatted, which seems to be a requirement for it to work with the new Android version.  After that headache, I transferred my maps and mapsVector directories back to the SD card and set about making them work with Locus.

Well, obviously, this was only the beginning of my troubles, thanks to Google's "scoped storage" feature

So, I cannot use adb to push my map files into the Locus data directories (because "security"...).  The blanket import functionality in Locus (taking all of the maps at once) also seems not to work.  Finally delving into the official documentation, it looks like I need to import maps one at a time from the maps manager.

This is where I am currently stuck.  I can import maps which consist of a single file, no problem.  But the map I really need is a ~20 gigabyte GEMF which is split into 11 files. If I select the first file in the sequence (the .gemf file), that gets moved into Locus' data directory and appears in the map list.  But it is missing 90% of the data, which is in the .gemf-1, .gemf-2, etc. files.  I cannot find any way to make the importer take these files too.  The file selection dialog does not allow multi-select, only one file at a time.  And importing a .gemf-n file by itself does not work because it is "not a map file."

Any help would be appreciated.  I had no idea this would become such an ordeal     Thanks google...

As I think about this more, it may be a job better suited for a GIS package like QGIS.  What I am trying to do is figure out exactly how high above river level a house is.

But I do think it would be neat if smartphone mapping software could take the new breed of LIDAR data and, e.g., generate custom contours with half-meter resolution.  Not very useful in mountainous regions, but in flatter areas, it could be.  Bicyclists in particular would appreciate more elevation resolution than the SRTM data gives, I think.

what is the problem with the SRTM-1 data format? Does it have a too low resolution for your usage?

Hi Menion,
I have not delved into the SRTM formats yet.  Is the spatial grid fixed in size?  These new datasets are much more finely gridded than the SRTM data.  1x1 meter is required at a minimum, although 0.5x0.5 meter would be better.  Height resolution is about 0.1 meter.
Mark

It seems .hgt is not a particularly flexible format.  GDAL can read it, but also says

The driver does support creating new files, but the input data must be exactly formatted as a SRTM-3 or SRTM-1 cell. That is the size, and bounds must be appropriate for a cell.

I was not able to find more details in a couple minutes of searching, but I can dig deeper.  It sounds, however, like it may not be possible to cram modern, high-resolution digital terrain models into this ancient format.

I suppose the next best thing (quite a distant second!) is to generate color relief maps from the geotiffs and use those as map overlays.  The problem is that in any given map view, I want to distinguish a total surface relief of only a few meters.  So, e.g. 200 meters would be at one extreme end of the scale, and 205 meters would be at the other extreme.  If I move to another place on the map where the elevation range is 300-305 meters, then my pre-generated, color-coded relief map is useless...

Apologies if this has been covered in the online documentation.  I searched for a while and could not find anything.

I have some very nice LIDAR data for my region (Washington, USA) which I would like to use within Locus.  The resolution is about 1m or a bit finer.  It is supplied as a GeoTIFF raster.

I could make images from this data using GDAL (color relief maps or hillshade maps).  But it would obviously be a lot better to have it integrated properly with another map.  Any suggestions for how to achieve this?  Is Locus capable of taking a raster format like GEMF and displaying it as elevation data instead of image data?  Or is there some format specific to DEMs which I should be targeting?

Thanks,
Mark

Perfect---thanks!!  Will let you know how it works when I get back in May.

[edit]

I'm writing a Perl script to take GPX navigation files exported from the excellent Dutch Fietsersbond Routeplanner (routeplanner.fietsersbond.nl) and convert them into GPX files I can use to perform navigation in Locus.  This is going to be pretty easy, as I can already tell a lot from studying the navigable tracks exported from Locus (e.g. after a mapquest navigation).  But there is one thing I need:  the list of codes in the <locus:rtePointAction> tags.

These are integers which encode what you are supposed to do at a navigation waypoint, e.g. turn right, turn left, go straight ahead.  I can reverse-engineer some of them, but I hope that by asking here Menion can provide the complete list and save me a lot of effort  

Thanks in advance.

edit
Menion, are you there?  To clarify, all I need is the enum, copied and pasted from the relevant source file.  I can figure out the rest.

Sorry for bumping my own topic but I leave for my trip in a few days...

Success!

Over the weekend I generated topo mosaics covering half the state (about 800 source GeoPDFs), tiled at zoom levels 9 to 16.  On my phone's screen (Samsung Vibrant), zoom level 16 is somewhat zoomed-in compared to the paper maps, but still sharp thanks to the new GeoPDF scans which are typically at 500 dpi--much better than the old DRG GeoTIFFs.  Panning and zooming is still snappy, even with the map split across several 2-GByte GEMF files.

This is a good way to find errors in the USGS georeferencing.  I've found at least two quads with the wrong coordinates, and sent feedback to the people at the historical maps collection.  Their database is also missing ~30 quads here and there across the state, which I hope will be filled in this summer.

I think I can probably fit the entire state in about 20 GBytes.  I am using JPEG tiles for considerable space savings over PNG, with very little loss in quality.

Hi Menion,

From my perspective the actual limit should not matter that much.  I will be making maps much bigger than 2 or 4 GB anyway.  But the GEMF spec defines how to split one map across files like foo.gemf, foo.gemf-1, foo.gemf-2, and so on.  Does Locus support that part of the spec?

We had discussions here in the forum: 2GB for Android 2.x, 4GB for Android >=3.

Thanks.  The whole state of Washington will probably come in near 32 GB, so I may split into west/east.  GEMF has support for splitting a single map across multiple files.  I hope Locus' GEMF implementation supports that...?

Hey, I'm back with an update on my USGS topo-mosaicking project.

I ended up going with the GEMF format, just because it is documented, and RMAP really isn't.  Also, there was python code provided to generate a GEMF file from a directory structure containing the raster-tile "pyramid."

I am starting out with the "historic" USGS GeoPDFs, which are the same maps as digital raster graphic (DRGs), but re-scanned at higher resolution and archived in one place for the whole country.  I had to get the complete list for the state of Washington, sort it in a spreadsheet, then write a script which attempts to pick out and download only the most recent version of each 7.5' quad.  This is an error-prone process, because many of the names are not perfectly consistent over time; for example, "Mountain" becomes "Mtn" or "Mt" and so on.  So it requires some manual intervention...

Next, GDAL tools are used to extract the rasters from each GeoPDF at 400 dpi.  At the same time, the "neatline" from the GeoPDF metadata is used to remove the map collars, and the maps are warped from whatever projection/datum they had in the GeoPDF, to the google-maps standard EPSG:900913, with the result saved as GeoTIFF.

Then, I use a modified version of the gdal2tiles.py included in the GDAL distribution to simultaneously mosaic a bunch of these GeoTIFFs and save the result as a tile pyramid at zoom levels 10-16.  (Modified, because out of the box it only supports PNG tiles, and I want JPG--much smaller with negligible loss in quality.)  This is the interesting part, as I have only tried a set of four quads; my goal is to mosaic ~1000 of them (the whole state).

Finally, a modified version of the python code on the GEMF web page takes the tile-pyramid directory structure and packs it into a GEMF file.  Transfer to my phone, and voila!  It performs great in Locus!

So the whole thing is done using open-source tools, and best of all, it's entirely at the command line--no GUIs.  Scriptable FTW  

I hope to have some larger GEMFs to play with in another week, and will report back here.

Does anyone know the file-size limit on an Android-formatted SD card?  Is it 2 GB, 4 GB, or something else?

Mark

Astronomer, eh?  That's my hobby too.  Lucky you, in Gran Canaria!

The way it seems to work here in the US, at least for topographic maps, is

1)  The data are free, but sometimes inconvenient to obtain, as the govt doesn't like spending money on fast pipes.
2)  Commercial entities take the free data, and convert to a proprietary format that only works on their GPS, or in their software (e.g. National Geographic TOPO!).  They also make sure their HW and SW won't read the free formats.
3)  Profit!!

(As you can probably tell, I don't have respect for this business model.  I think it is fundamentally immoral.)

My starting premise is usually that the data are free to begin with.  When this is not true, then the rules change...  e.g. it used to be like that for roadmaps, but as you noted OSM is dragging nav data into the free world too.  Good times!

How much longer do you think desktops will live? The sooner you cut off all dependencies, the better you'll be prepared...

I'm very much a "live free or die" kind of person, so personally I hope the desktop, or something like it, lives forever.  I like my phone, and I'm impressed with my parents' iPad, but if the walled-garden / app-store model of computing is our only future, we will have lost something profound.

(Please note, when I say "free" I mean free in the sense of freedom, not in the sense of price.  I have no problem paying for quality software like Locus Pro, but I don't like proprietary interchange formats, so I don't care what TTQV4 or Compeland happen to use, unless those formats are open, like geotiff.  And, top priority should go to supporting those formats used in open-source software like GDAL, because that ultimately makes the world a better place.  BTW, geopdf seems to be going the way of pdf, i.e. an initially proprietary format shoved down our throats but eventually standardized and open.  All of the $$$ for one company while sort-of keeping the moral high ground  :roll: )

If you're interested in some radical speculation on the future of mobile vs desktop computing, I highly recommend Cory Doctorow's recent keynote at the 28th Chaos Computer Conference in Berlin, "The Coming War on General Purpose Computation":

http://www.youtube.com/watch?v=HUEvRyem ... r_embedded

Sorry for the tangent.

I dunno, Joe, I still think we're going to pay for that concept in battery life.  That always ends up being the limiting factor.  Maybe if we get another order-of-magnitude improvement in energy density...  It will happen, eventually, although I do worry about the safety implications.  Imagine 50 watt-hours (that's 180 kW-seconds--imagine it) of energy sitting in your pocket.  It's a bit frightening.

Re: GEMF vs RMAP, it seems like GEMF could be extended to support other projections.  Until someone documents and standardizes "open RMAP" I am too nervous to use it.

If you want a third possibility, I would suggest taking a look at spatialite and rasterlite (the tiled-raster extension of the standard).  These are at least supported in GDAL, which instantly elevates them into standards, because the GDAL library is kind of universal in GIS software.  They are sqlite formats of higher complexity than the simple sqlite Locus uses today.  I don't know enough about it to say if it solves the speed problems, though.