Locus Map - forum

Support => Troubles & Questions => Topic started by: noisyriver on March 07, 2013, 14:43:52

Title: .sqlitedb as Overlay not working?
Post by: noisyriver on March 07, 2013, 14:43:52
Edit 2
Narrowed the problem: Locus is not reading this map when tile recalculation and JPEG compression is activated in MOBAC.

I exported four files with MOBAC:
1. A normal OSM map area with compression and recalculation to Rmap sqlite: No problem in Locus
2. Hike & Bike Map hill shade layer to orux sqlite: no problem in Locus (but not possible to select as overlay)
3. Hike & Bike Map hill shade layer (same region) without compression and recalculation to Rmap sqlite: No problem in Locus
4. Hike & Bike Map hill shade layer (same region) with compression and recalculation to Rmap sqlite: Empty (grey) in Locus

---------------------------------
Older Message:

I have a problem:
I want to create an offline hillshade overlay for my offline OSM-Vector-Map.
So I have downloaded me a small region of the Hike&bike map hillshade layer, zoom 8 to 15, via MOBAC in a RMAP .sqlite file.
I am able to open this layer alone in Locus and see the hillshading very nicely.
I am also able to add this map as overlay with the overlay function. But it is not displayed, the OSM map is still "flat".
Doing the same directly with the online Hike&Bike map through Locus works fine, I have a nice hillshading. For testing I also copied the .sqlite file Locus was creating from downloaded tiles and used this as a offline map and this was working fine too.

But no success with the .sqlite from MOBAC.

(MOBAC 1.9.7., newest Locus Pro, Android ICS on an Galaxy Note 2.)

Thanks for any help!

edit
Invested some more time and found that Locus shows the sqlite file empty (grey) also when viewed alone. (Had the Hike&Bike hillshade overlay on by accident). Made some tests with other formats (paper atlas and export to Orux) and also exported other sources with sqlite from mobac to Locus. No problems. Only the Hillshade overlay is not shown only in Locus.Any ideas?
Title: Re: .sqlitedb as Overlay not working?
Post by: kech61 on May 30, 2013, 18:08:27
Hi,

All I can say is that I can confirm this
In addition if you try to reprocess the tiles as png (8/4bit) the screen is black.

The unprocessed version is IMO useless because of the size of the resulting file.
@Menion, is there any way to get a reprocessed (small) offline hillshading sqlite file working in Locus?
Title: Re: .sqlitedb as Overlay not working?
Post by: Menion on May 30, 2013, 18:30:22
I'm not familiar with mobac, so may you guys create for me two small maps (just for example four tiles) as noisyriver describe as map 3. and 4.

3. Hike & Bike Map hill shade layer (same region) without compression and recalculation to Rmap sqlite: No problem in Locus
4. Hike & Bike Map hill shade layer (same region) with compression and recalculation to Rmap sqlite: Empty (grey) in Locus

so I should compare where are main differences in stored format and try to fix map 4. thanks
Title: Re: .sqlitedb as Overlay not working?
Post by: kech61 on May 30, 2013, 20:18:55
Quote from: "menion"so I should compare where are main differences in stored format and try to fix map 4. thanks

Hi Menion,

I made 3 Versions, zoomlevel 10-14:

uprocessed
jpeg 50% compression
png 8bit (the smalest file by far, ...if OK)

Permalink: http://www.openstreetmap.org/?lat=48.29 ... 0&layers=M (http://www.openstreetmap.org/?lat=48.299&lon=15.557&zoom=10&layers=M)
Zip-File: www.openandromaps.org/files/shade_test.zip (http://www.openandromaps.org/files/shade_test.zip)

Thanks for all your efforts,
Christian
Title: Re: .sqlitedb as Overlay not working?
Post by: Menion on May 30, 2013, 20:39:58
Christian, I thanks you ...

anyway I'm quite confused now. File ShadeTest_png8bit.sqlitedb seems to contain 1364 tiles where all are just 141 bytes big images. In all programs I tried, it display just completely black tile. Are you sure this works? Because 141 bytes is quite low on tile where should be at least something ... (and all, tested 3 from different places, are same 141 b)
Title: Re: .sqlitedb as Overlay not working?
Post by: kech61 on May 31, 2013, 08:49:23
Quote from: "menion"Christian, I thanks you ...

anyway I'm quite confused now. File ShadeTest_png8bit.sqlitedb seems to contain 1364 tiles where all are just 141 bytes big images. In all programs I tried, it display just completely black tile. Are you sure this works? Because 141 bytes is quite low on tile where should be at least something ... (and all, tested 3 from different places, are same 141 b)

Hi Menion,

I just tried to work with a original tile (with lots of elevation data) from tileserver and optimized it in Fireworks.
original file: PNG 43k
jpeg50%: 7k
optimized png (palette optimized, Alphatrans, png8, bells n' whistles) = 10-14k

So the png8-file from Mobac is definitly broken and the way to go is IMO the optimized (shrinked) jpg.
I hope that the 50%jpeg is OK out of Mobac.

The overall size of the resulting sqlite-db seems to vary between the different sqlite-formats too.

I will do some further testing and report.
Title: Re: .sqlitedb as Overlay not working?
Post by: kech61 on June 02, 2013, 14:57:16
Hi Menion,

I will stop this (hillshading with Mobac) and switch to Maperitive - I'v read that Locus support MBTiles sqlite-format > is this Format capable to support transparent hillshading as overlay in Locus?
Actually I cant provide an example - I'v to learn to drive Maperitive  :roll:

Best regards,
Christian
Title: Re: .sqlitedb as Overlay not working?
Post by: Menion on June 05, 2013, 15:32:27
Hello Christian,
  I played a while with Maperitive. After around 4 hours it created complete hillshade map for czech republic in levels around 10 - 15 and size around 200 MB, not bad. Anyway I just tried to display it and seems that it do not work. Locus is trying to load images but without success. I'll try it later anyway all I do inside locus is using function "Bitmap.createFrom(byte[] data)" ... where byte[] is just array of bytes and Android itself handle it and create bitmap image from raw data. So there is not much space for some improvements. I'll check it anyway once more
Title: Re: .sqlitedb as Overlay not working?
Post by: kech61 on June 07, 2013, 12:12:54
Quote from: "menion"Hello Christian,
  I played a while with Maperitive.

Hi Menion,

After some testing with maperative I can say that it works.
You have to set map-background-opacity  to "0" in the ruleset and reaply it right bevor generating Hillshade (the ordinary hillshade, not the Igor-relief)

The problem ist that Mobac destroys the transparency of the tiles when trying to shrink them.
So the Tiles have to generated and processed within the tiles/-Folder one by one, preserving transparency,  _bevore_ feeding them into mobac.

HUHU..  :o

This means for the Alps processing of about one millon tiles (Level 8 to 15).
I wrote a script for parallelization of this, its curently running, an running and...

The tiles/-folder for the Alps (Level 8-15) is as heavy as 9GB,  :shock: I hope to shrink it to >2GB

For a small area I tested this and it works = maperative>shrinking Tiles>Mobac>RmapsSqlite>Locus, however its real heavy number crunching.

Edit: I tested mbtiles-Format right out of Maperitive and it worked, however there is noch chance to shrink it so: not the way to go.
Title: Re: .sqlitedb as Overlay not working?
Post by: Menion on June 07, 2013, 12:37:41
hmm these are terrible numbers. I was really happy when I firstly saw your file with shading. File ShadeTest_png8bit.sqlitedb .. it looked that it's possible to create so small file with shading :) nevermind ...

I have similar result as you. 200MB just shading for Czech, isn't bad. 9 GB for Alps, is worst ... also on second side it means another around 4 MB of RAM consumed on average device (as an overlays), also speed reduction etc ... I think way to go is with "Locus internal shader", it just needs to improve little bit. Did you tried shading of vector maps in Locus?

If not, check Locus configuration file http://docs.locusmap.eu/doku.php/manual ... ation_file (http://docs.locusmap.eu/doku.php/manual:configuration_file) and last parameter dev_enable_vector_maps_hillshade set to 1. Locus will then use HGT files and generate "shading" on vector maps ...

EDIT: with latest 2.12.2 version it really isn't so bad :)
Title: Re: .sqlitedb as Overlay not working?
Post by: kech61 on June 07, 2013, 14:04:33
Quote from: "menion"If not, check Locus configuration file http://docs.locusmap.eu/doku.php/manual ... ation_file (http://docs.locusmap.eu/doku.php/manual:configuration_file) and last parameter dev_enable_vector_maps_hillshade set to 1. Locus will then use HGT files and generate "shading" on vector maps ...


Hi Menion,

Always a step ahead  :D  , isnt it  ;)
I tried your solution with viewfinder .hgt's and for me this is deffinitly the way to go - thanks for bringing this Beta-feature to my attention, it will save a lot of time for walking, cycling etc. instead of sitting at the computer - THUMBS UP!
Title: Re: .sqlitedb as Overlay not working?
Post by: Menion on June 08, 2013, 08:39:18
Hello,
 not always :). It's still not perfect. Main problem for me is that I still use software rendering on Android (not OpenGL) and there are some issues, anyway as you wrote - this is the way to go. As we both know, sizes of shadings images are quite crazy and this is very elegant solution where 3' HGT files are enough, so not so much space is needed.
Title: Re: .sqlitedb as Overlay not working?
Post by: joze on June 11, 2013, 22:23:09
Hi,

I tried dev_enable_vector_maps_hillshade and it seems to me that it generates a very weird blotchy hillshading where especially small gradients are over-exaggerated.

Here's a comparison.

Left: openandromaps.org vector map with locus hill shading (by dev_enable_vector_maps_hillshade). height data is dem3 files from viewfinderpanoramas.org
second: alpha.map1.eu raster map with it's own hill shading
third: hikebikemap raster map with it's own hill shading
right: hill shading created myself from dem3 files from viewfinderpanoramas.org by maperitive and mobac

Thus at least the left and right images use the very same source dem3 files from viewfinderpanoramas.org

As you can see, small gradients are largely exaggerated by the locus built in hill shading, see for example red arrow which points to a region with low gradients. As I know the displayed region, I can say that the raster images correspond to reality MUCH better than what is generated by locus.

--> what algorithm do you use to display the height data?
Title: Re: .sqlitedb as Overlay not working?
Post by: gynta on June 11, 2013, 23:12:50
compare?
oh yes! let's  compare (//http://www.oldboys.at/dl_tmp_pix/hillshades/)  :mrgreen:

means, Locus is not soo bad...
Title: Re: .sqlitedb as Overlay not working?
Post by: joze on June 11, 2013, 23:43:33
Quote from: "gynta"compare?
oh yes! let's  compare (//http://www.oldboys.at/dl_tmp_pix/hillshades/)  :mrgreen:

means, Locus is not soo bad...

Hi,

thanks for this exhaustive comparison -- but still I think that all example show that locus' built-in hill shading does it sub-optimal.
Title: Re: .sqlitedb as Overlay not working?
Post by: Menion on June 12, 2013, 05:20:08
it's not optimal, that's why it isn't officially in public release, I'm also not satisfied with it ...

algorithm? own :). I was unable find and working solution so I created own. It's not best, but I don't have better ...

Gynta created very nice comparation, so I'll use it and tune shading a little more ...
Title: Re: .sqlitedb as Overlay not working?
Post by: joeloc on June 23, 2013, 13:23:51
I quite like the builtin hillshading from hgt... good job!
Title: Re: .sqlitedb as Overlay not working?
Post by: Menion on June 25, 2013, 06:12:17
:) thanks. I'm not yet satisfied to publish it, but it's a good way to go I think ... if it will work, I hope I'll be able to apply it on every map type, not just vector
Title: Re: .sqlitedb as Overlay not working?
Post by: jjensson on July 15, 2013, 04:34:30
Hi Menion,

This overlay shader stuff is genius! But i noticed that the hillshading takes away much of the contrast of the map. I thought it was some error on my part, but now i notice that your screenshot has the same problem.

May i suggest that you use a different blending mode for the shader overlay, a method that does not lighten the darks (text, lines, etc). In Photoshop terms maybe Soft-Light + a less bright hillshade overlay? Just poking in the dark here... As a brute force method the shader could amplify the contrast a bit, but i don't know if the processing cost wouldn't be too high.

Locus rocks!  :ugeek:
Title: Re: .sqlitedb as Overlay not working?
Post by: Menion on July 15, 2013, 12:14:55
I spend quite a lot of time by improving this hillshade system and I think it's going to be ready to publish now :) At least you'll see in next version
Title: Re: .sqlitedb as Overlay not working?
Post by: jusc on July 15, 2013, 12:51:39
Can anybody proof, if you can  set
dev_enable_vector_maps_hillshade set to 1
and later set it back to
dev_enable_vector_maps_hillshade set to 0?


Locus crashed after zoom in or out. I had to delete this line completly to get Locus to run again. (Maybe it´s only my phone)
Title: Re: .sqlitedb as Overlay not working?
Post by: gynta on July 15, 2013, 13:06:58
hm works here.

start
locus vectormap - locus cycle theme
quit
changed to 1
start
move map
quit
changed to 0
start
move and zoom map
quit

...with both sgs2 devices.
Title: Re: .sqlitedb as Overlay not working?
Post by: jusc on July 15, 2013, 13:24:06
Did you try zoom in and out? You wrote "move.." And was hillshading definitiv displayed and later not displayed again.
After stopping Locus and changing the settings on SGS2 back to 0 (zero) and restarting Locus the hillshading ist still visible, but Locus doesn´t crash.
The crashs are on the Note2, but this is probably another problem..
Title: Re: .sqlitedb as Overlay not working?
Post by: tommi on July 15, 2013, 16:52:17
For me the hillshading on and off on vector maps works. If enabled, hillshading is displayed from zoomlevel 9-15. In level <=8 the world map is displayed, from zoomlevel >=16 no hillshade is not displayed.
Title: Re: .sqlitedb as Overlay not working?
Post by: jjensson on July 15, 2013, 19:34:21
Quote from: "menion"I spend quite a lot of time by improving this hillshade system and I think it's going to be ready to publish now :) At least you'll see in next version

Can't wait to see it in action - i have all of Europe in .hgt-files on my extSDcard. I knew i bought the right app for my mapping needs, and i only buy the best ones.  :)
Title: Re: AW: .sqlitedb as Overlay not working?
Post by: jusc on July 15, 2013, 20:04:16
Yes, I tried it again. It's ok on SGS2. The other phone does have a problem.
Thanks for double check.
Title: Re: .sqlitedb as Overlay not working?
Post by: jjensson on August 12, 2013, 23:57:35
Is it possible to give a coarse ETA for the next update? I'm looking in the app store almost every day but can't find anything.

EDIT: Never mind, i found a bunch of updates. Turns out, Google Play doesn't show updates for Locus, go figure...