sqlite to GEMF map conversion tool

[stebu]

This is a small PC utility that enables to convert and combine several sqlitedb maps (downloaded from your Android /Locus/maps into PC) and other GEMF maps into one (possibly HUGE) GEMF map.

GemfTool information and download

Enjoy!

Nice comments are wellcome on this board :roll:
-stebu
(edit: 30.10.2011 Additional functionality, ne name)

[stebu]

Version beta.002 released at the same link.
Added some string length checking for crazy inputs.
Added check that the header table fits completely inside a maximum (2Gb) file.

I tested by creating a combined map from Amsterdam and Helsinki at zoom level 14. It's a foolish effort, header contains over 600000 empty tiles (7.2Mb)! Took a couple of minutes to create. Not recommended practice, but it worked.

  zoom, min x, max x, min y, max y, (act/area)
    14,  8410,  9323,  4701,  5391,     (157/631574)

[stebu]

I skipped a version, it's now 004.

The tool has been renamed to GemfTool(.exe). It can now combine Locus' sqlite maps and gemf maps (and a mixture of them). The empty tiles (e.g. between distant maps) can be removed by the map combination process. It can read and write huge maps in 2Gb chunks.

Enjoy!

[stebu]

I had some False Positives by Avira. I don't know if they have disappeared now, in V006?

Anyhow, the previous versions had some indexing problems, the maps were created, but the tiles were "a bit" out of order, sorry  :oops:

[Menion]

stebu, I updated Avira to new version and still reporting error. But it's probably too early to receive their update. Anyway leave it be, I'm really not worried that there is some virus . And thank you very much for your work. I'm sure that after some propagation, it will be more then useful tool. Also, if you want to propagate it more on new Locus homepage, I may create special page in support menu if you want ..

[tomtomxp]

is there a tool, to extract sqlite maps to bmp or ecw or any typical raster map to use/mod with global mapper or compegps??

[stebu]

is there a tool, to extract sqlite maps to bmp or ecw or any typical raster map to use/mod with global mapper or compegps??

Hi!
If you can use a Python script, I have one that reads Locus's sqlite maps and sends the tiles to disk files. There is no image conversion, so you'll get .png or .jpg files.
Let me know, if you are interested.

[tomtomxp]

Hi, yes please ive heard of python script but might need a lil direction in executing it, cant be that hard if you know the command/script.
thanks

[stebu]

Hi, yes please ive heard of python script but might need a lil direction in executing it, cant be that hard if you know the command/script.
thanks

Yes, Python is amazingly powerful, yet easy to use (imho). The biggest problem I had was with changing my text editor to use spaces instead of tabs  8-)
Python site where you can download Python. My script uses the older version, so please use Python 2.7(.x).
You'll also be needing Sqlite, this you can get from //http://code.google.com/p/pysqlite/downloads/list , the file pysqlite-2.6.3.win32-py2.7.exe  works for me. First install Python, then execute the downloded installer pysqlitexxxxx.exe.

the python scripts are zipped on my web-site. There's a bat file with the "keep" parameter, this leaves the sql-tiles in their folders as .png or .jpg files.

This should keep you busy for a while  

[stebu]

My ISP shut down my home page.

Now you can (hopefully) get the conversion tools (V006) from Dropbox:

https://dl.dropboxusercontent.com/u/64901427/GemfToolV006.zip

The zip contains:
gemftool.html that has some description
readme.txt probably the same content as above
gemfread.exe
gemftool.exe

Some false positives have been reported by Avira. But I'm 99.9999% sure that they really are FALSE.
One ppm is reserved for NSA

[Bucky Kid]

Hi stebu, thanks for this tool. I wanted to ask what density and number of rounds should I use to get max compressed map. Have these values impact on performance?

And second question is why gemftool creates two sets of map:
For example I have 3GB big atlas named atlas.sqlite, After conversion I get atlas.gemf, atlas.gemf-1, atlas_.gemf and atlas_.gemf-1. What's the difference between maps suffixed by underscore and wihout it?

[stebu]

Hi stebu, thanks for this tool. I wanted to ask what density and number of rounds should I use to get max compressed map. Have these values impact on performance?

And second question is why gemftool creates two sets of map:
For example I have 3GB big atlas named atlas.sqlite, After conversion I get atlas.gemf, atlas.gemf-1, atlas_.gemf and atlas_.gemf-1. What's the difference between maps suffixed by underscore and wihout it?

It's over 2 years since I made or used Gemftool! So my answers might not be precise.

The more compressed (empty tiles squeezed out) the map is, the larger the header part is; more ranges of smaller map areas. I think the performance impact is on the app that uses the Gemf map, and I have little experience with this.

If you find that the Gemftool itself is running a long time, then reduce the number of rounds directly and/or reduce the goal density.

The files with underscores are temporary files and  may be deleted. In yor case atlas.gemf, atlas.gemf-1 are the result of the conversion. There's two files, because the size of the map exceeds 2GB, 3GB map is split into two files.

[onelook]

Can I find this tool anywhere now, please? Dropbox link is expired.

[Blubbo]

I emailed the author and thanks to him and Al the tool is available again
You can find it here: https://www.cgtk.co.uk/gemf