Locus Map - forum

Support => Wishlist => Implemented => Topic started by: Ferretty on July 23, 2011, 22:09:40

Title: KMZ/KML POI improvements
Post by: Ferretty on July 23, 2011, 22:09:40
I just played for a few days with the free version of Locus and, amongst others, also imported some POIs from KML/KMZ files. Though that's great stuff, I'd suggest some improvements for this:

Map sources to POIs
As it is now, one imports a KML/KMZ into a category. Multiple sources (i.e. KML/KMZ files in this context) can be imported into one category this way. But if one finds a better source for parts of them, or want to get rid of some for some other reason,, it is "all or nothing": One can either delete all POIs of a category, or none. Or one has to delete each POI separately, which can be quite annoying.

Now, if the source would be stored along (and here I mean the file name), one could remove all POIs of a given filename.

Multiple files in one KMZ
KMZ files are not simply a "zipped KML file". Sure, they can be this -- but they can be much more: One could include icons and images this way. And as Locus supports an offline mode, those additional files could be used along. For example, not only a textual description for a POI (using the KML <description> tag), but, say, a photo of the location, contained in the description using a relative path with an <IMG> tag. And for sure one could use specific icons.

With that, I could provide some of my KMZ files for the POI sources here. If an example KMZ file is needed, just let me know and I will provide it.
Title: Re: KMZ/KML POI improvements
Post by: Menion on July 24, 2011, 12:57:35
Hi,
  I'm not sure if I understand first part. If you import any KML file that contain more points (which can be separated by own styles), you can then in "Point screen", filter, select and delete simply only points you don't want to have ...

  about extended support for KMZ files, some sample file should be welcome (best on email locus@asamm.cz), thanks
Title: Re: KMZ/KML POI improvements
Post by: Ferretty on July 24, 2011, 15:21:39
For the first part: Imagine you imported 1.250 POIs from 3 files into 1 category. Now it turns out the POIs of one of those files were crap, and you have to delete 375 POIs. Did I miss anything -- or is the only possibility currently to mark all 375 by picking them manually, and then remove the marked POIs? In that case, wouldn't it be much easier to say "Mark all POIs imported from crap.kml", and then delete them? Or maybe move them to a different category?

About the second (extended KMZ support), you should have mail in your inbox (or the Spam folder, not sure  :D )
Title: Re: KMZ/KML POI improvements
Post by: Menion on July 24, 2011, 15:31:08
I don't see any advantage in creating some tool on your first part. Why should I delete these points, when I can fix the file and import and overwrite wrong points? Or I can delete whole category and import only valid data again (it's question of minute). I don't just see anything useful on adding "something that allow delete data based on source of import" ---

thanks, mail receive, I'll look at it during week
Title: Re: KMZ/KML POI improvements
Post by: Ferretty on July 24, 2011, 16:02:24
Hm, well, it was just a suggestion. I could e.g. no longer have the file. Or do not want to/know how to edit it (I know how to, but other people might not). So if you are far from your PC, and no longer have the original KML/KMZ files on your device... I think e.g. of a "cleanup" half a year later, when you got everything overfilled and a bit messed up, got used to Locus, and think of better organizing your POIs. Would you remember which file you imported what POIs from after half a year? -- But as said, it's just a suggestion and you are the boss  8-)

As for the second part: Great! I'm looking forward to your next response on this  :P
Title: Re: KMZ/KML POI improvements
Post by: estein on July 25, 2011, 20:10:51
Hi, I'm also very interessted in the "second part" !!!

I'm just planing the vacation with collecting and preparing maps and POIs and so on for offline usage.  
The support of KMZ with integrated images would be a great help!
E.g. campgrounds with text description and a picture copied from the web or "must see places" with description and image as appetizer ...

At the moment a KMZ ist displayed as text but the included image is just an empty border ...  :-(

best regards, estein
Title: Re: KMZ/KML POI improvements
Post by: Menion on July 25, 2011, 21:43:36
hmm I'm just curious, how to handle with image data. Copy then to cache folder or import them directly into database? Hmm ... what would you suggest?
Title: Re: KMZ/KML POI improvements
Post by: Ferretty on July 25, 2011, 23:57:28
The latter. In combination with my first suggestion. So if the user selects to delete all data from the imported KMZ, connected images will be deleted along. Of course you could connect images also directly to each POI if using the database. But placing images into the database would add unnecessary size overhead IMHO.

But as for the connection (between POI/KMZ and images): It could also be that multiple POIs share an image (e.g. with styles for the icon that will probably be the regular case), and the user decides to only delete some POIs. So you will need a kind of "counter" for how often an image is used. If that counter goes to zero, the image is no longer in use and can be deleted.
Title: Re: KMZ/KML POI improvements
Post by: estein on July 26, 2011, 15:46:05
I would also suggest to import the whole KMZ into the database.
In my understanding a cache is a temporal thing. If I import a KMZ (or POI) into Locus, I would expect the whole data to be safe there.
Title: Re: KMZ/KML POI improvements
Post by: Ferretty on July 26, 2011, 15:58:26
Maybe the word "cache" is a bit misleading here. The pictures should be saved in the "file system", which does not necessarily mean "temporary" (the database file itself is stored in the file file system as well). Having them stored there instead of in the database may save some overhead (though I'm not sure how much that is), and has the side-effect that you can browse it (and thus the images) with every standard browser to view the images.
Title: Re: KMZ/KML POI improvements
Post by: Menion on July 27, 2011, 20:41:47
uf, it was much worst then I expect ... testing version is here viewtopic.php?f=25&t=782 (http://forum.asamm.cz/viewtopic.php?f=25&t=782)

system is "easy" if you want to know. I take KMZ file, all images copy into Locus/data/import folder. All kml files are then imported into database. Icons in Style and StyleMap tag are now correctly added to POIs and are taken directly from data/import folder. Same with images included in description in HTML code. I tested it all on file from Ferrentty, so please try any other and let me know how this works. Also count that all images remain in data/import directory even after you delete POI's and in this way it probably remain.
Title: Re: KMZ/KML POI improvements
Post by: Ferretty on July 27, 2011, 21:29:22
Thank you very much, menion! I will try that out ASAP.

So how do you suggest to clean up "old images" when POIs are deleted? Because with no such option, the folder may fill up with time proceeding (consider people just importing POIs for vacation, and dropping them lateron, importing new ones for new vacations, and so on).
Title: Re: KMZ/KML POI improvements
Post by: Menion on July 28, 2011, 06:03:34
it's little bit different to handle these images

there are two types. One are used in Style (Href) tag, so they are needed for poi icons. There is not much problem with them. Just small images (in your case only one small image). I can do something with url to store them. Maybe put them into cache/images directory as other small images.

Second are much worst. Url to some big images is inside "description" tag of every POI and I cannot work with this. System itself determine url from which load images. If url starts with http, that sure (downloading from web - same with geocaching listing), anyway when there is some relative url, it try to load image from local storage. And I define this storage as Locus/data/import folder. And that's all. So to sumarize it - I have no influence on images that are attached in description of POI. I can only place images on place where can be found when needed. And for now it seems for me almost impossible to check for every deleted poi, if there is not some link to relative local storage and some image there, etc.

  Hope you understand. Anyway my suggestion is to simply delete data/import directory sometime. I'm also thinking if not to place this directory into Locus/cache directory but these data are not as temporary as rest of cache directory, so it will rather stay in data folder (to be more permanent)
Title: Re: KMZ/KML POI improvements
Post by: Ferretty on July 28, 2011, 07:48:27
I fully agreee as to the Icons: Just small stuff. As for the images: I understand the problem (as I wrote initially, I'm not sure this approach is 100% conform to the standards: It is not mentioned in the API reference. Obviously because it is not directly part of the KML data, but -- as you already pointed out -- the description).

That's why I suggested to save the KML name along. If there'd be a "marker", one had something to check against (e.g. use a "cleanup" button to check for unreferenced images, which is hard to do with the IMG tags from descriptions). So if your other idea (storing the images together with the POI in the database, so there is a direct connection between POI and image -- just for the big ones, of course, as the icons are set for the entire KML and not a single POI) would make things easier (as you don't need to remember the connection), it could be worth a try. But then you had the problem with where the relative URL points to, I guess... which could be solved by placing them into the cache additionally, and have a button somewhere to place them there again in case the cache was purged. Gnaa. Sounds like a huge overhead (at least in size, as this way you need to store the images twice)...
Title: Re: KMZ/KML POI improvements
Post by: Menion on July 28, 2011, 07:55:19
hmm, main question for me is if the work on this worth it. Actual solution works fine and have only one disadvantage. When you'll do a lot of KMZ (with images) import, you'll have many images in Locus/data/import folder. Question is whether this is really a big disadvantage. I don't think so. Every at least a little experienced user, can delete this directory. Also when I import any data that I worth it, I keep them save for later.

Storing data into database isn't good idea. When you'll have more images in few points, I'll save some data twice. Also url to image is in some description so I'll have to parse description of every point if there isn't any relative path to image ... ehm ... terrible. And this will also affect speed of import.

No sorry, I think that best will be to leave it as is now if all works fine. After few weeks of usage, we'll see if there is need for some improvements and if stored images are really a big problem, posssible?
Title: Re: KMZ/KML POI improvements
Post by: Ferretty on July 28, 2011, 08:15:43
100% agreed. I'll keep my mind busy to think of an easy approach. Would it e.g. be possible to search the descriptions for the existing images? Just using some pseudo-code here:

foreach (image in /data/import/*) {
  SQL(SELECT...WHERE description LIKE '%image%')
  IF no_data_found rm -f image
}

So user could run this routine for a cleanup. Not 100% exact -- but if the image is mentioned nowhere in the desc, the connected POI must have been deleted (but if it is mentioned, it is not sure to be the IMG ref -- except when the SQL is modified correspondingly to match on something like "SRC='%image%'"). Or I have missed some other point where it could have been stored, not sure. It's just a raw idea to maybe keep in mind for later  ;)
Title: Re: KMZ/KML POI improvements
Post by: Menion on July 28, 2011, 08:32:52
something like this should be done later anyway it will be really slow. I store description data not directly as text in database but as a byte array in more complex structure, so I firstly need to load whole data, create from them object I need and then grab only description and search it. That's also reason why it takes so long to do full text search in locus. On second side it bring me much advantage so I decided for this method.

  Anyway we'll see :)
Title: Re: KMZ/KML POI improvements
Post by: Ferretty on July 28, 2011, 08:52:43
Yepp -- didn't mean to hurry you up  :D Maybe that could go to some "cleanup" button later. Better think it thoroughly and wait a bit longer than to have to do it twice  :lol:

Thanx a lot! I love this. During the next days I'll play a bit with my KMZ files and see to make some "special Locus versions" out of them. Just reworked the sample I sent you, as it used a 2-column table with the image next to the description, which is not quite optimal to be displayed on a mobile phone (one has to scroll horizontally and vertically). Now I placed the image inside the desc. Need to check this later today...
Title: Re: KMZ/KML POI improvements
Post by: Menion on July 28, 2011, 09:01:10
perfect! I'm looking forward to your work :). And when you do some simple description how to create such KMZ files, it would be more then welcome! (and absolutely agree with removing second column and place images directly into text)
Title: Re: KMZ/KML POI improvements
Post by: Ferretty on July 28, 2011, 09:47:38
Sure! I just had to leave for work, and my test device has no SIM in it and thus no network, so I cannot continue on this before I'm back home. I will check the new KMZ this evening and, if it looks OK, attach it to the other thread (were I already placed the original one) for everybody to check. If people like it and want more, I'll go over my collection  :D

concerning the "How to create...": Sure, I can write something down. Question is: There are multiple ways to do that. Easiest ist of course using Google Earth (plus some HTML template for the description), which is what I do. Guess I'll use this way for the Howto (in the appropriate forum section, of course)?
Title: Re: KMZ/KML POI improvements
Post by: estein on July 29, 2011, 15:06:51
I've tested the 1.9.4.2. Thanks for your work!

Loading the Munich example KMZ von Ferretty and some much simplier self created (only text and a <img src="...">)
=> everything works great!

But after a while the import of new KMZ failed. No pictures are loaded an even the already imported disappeared.

I found out that the folder "import" wasn't there any more! And the strangest thing is that even the manual
creation of it with Root-Explorer failed! (Locus was not running any more, checked with Taskmanager).

After a reboot, the manual creation of the "import" folder was possible again, but Locus is stil failing.
New KMZs are imported, but without pictures ...

All other function seems running and the parallel installed Locus Pro is also working.
Title: Re: KMZ/KML POI improvements
Post by: estein on July 30, 2011, 20:36:59
I think I found my problem: In the section <Style id ...> were still icons with a "http://..." reference.
After replacing them with local images it works!  

Thanks Ferretty for the manual which gave me the hint ...
And thanks Menion for the update  of the Pro Version!
(perhaps you could make the import a little bit more tolerant ;))
Title: Re: KMZ/KML POI improvements
Post by: Menion on July 30, 2011, 21:44:27
hmm this is really weird, because http in style tag is also supported!! Don't you still have your old file that you can send me for test?
Title: Re: KMZ/KML POI improvements
Post by: Ferretty on July 30, 2011, 22:04:02
It should make no difference if you mix them (http and local), right? But as for your "weird": there is one case which would lead exactly to that behaviour: If estein was OFFLINE when trying the POIs (i.e. having no data connection), the http: images could not work (no network connection to retrieve them). And I guess that's what was meant.
Title: Re: KMZ/KML POI improvements
Post by: Menion on July 31, 2011, 07:24:24
yes that's probably ... but just to let you know - online url to images are cached. If is http in style attribute, I cache image into cache/images directory. If this http is in description, it should be cached by http browser (that is used to display POI info)
Title: Re: KMZ/KML POI improvements
Post by: Ferretty on July 31, 2011, 10:40:56
Sure -- but is that done on import, or when you display a given POI? In the first case, the problem should not have appeared. In the latter, when I import a KMZ containing 1.000 POIs, I'd have to click them each one on my own in order to get them cached -- the ones I miss won't be cached. And even when done on import: If one is offline at that time, there's no way to cache anything  :lol:
Title: Re: KMZ/KML POI improvements
Post by: Menion on July 31, 2011, 18:02:17
it's done during import, so if you have internet connection during import, all http request that are in 'style' tag, should be cached. If you don't have internet connection, you have then click manually on every unique poi. Hmm as I think ... all http images should be loaded again when you display POI category with these images, so not a big problem
Title: Re: KMZ/KML POI improvements
Post by: Ferretty on July 31, 2011, 18:06:56
Just to make sure: External images in descriptions get cached on import already? Or when the corresponding description is displayed (on click on the icon in the map)?
Title: Re: KMZ/KML POI improvements
Post by: Menion on July 31, 2011, 18:16:33
only when displayed. It work in way that: when you click on any POI on map, screen you see is small web browser, so images are downloaded only when needed. Also they aren't stored in same cache directory of locus but in phone memory. This small browser have own cache that I cannot handle with. So if you have some POIs with http in description, you cannot be sure, that image will be still cached for example after few days. It's same (problem) with listing of geocache. You can have it full offline (so also with images) only when you display this listing.
Title: Re: KMZ/KML POI improvements
Post by: Ferretty on July 31, 2011, 18:28:08
OK -- thanks for this explanation! I was just confused, since otherwise why should I rework my POI collection to include the images  :shock: So now it looks like an additional feature: Local files probably won't be cached (I hope?) and thus don't eat precious internal (phone) memory.
Title: Re: KMZ/KML POI improvements
Post by: estein on July 31, 2011, 18:43:37
Hi Menion and Ferretty,

This little KMZ "kills" my files folder when importing into Locus without an internet connection.
Title: Re: KMZ/KML POI improvements
Post by: Ferretty on July 31, 2011, 19:03:50
I guess that's something for menion rather than for me -- KMZ files contain no "kill" command, just content. So I have no idea how to write a KMZ which does that...
Title: Re: KMZ/KML POI improvements
Post by: Menion on August 01, 2011, 15:49:24
Quote from: "estein"Hi Menion and Ferretty,

This little KMZ "kills" my files folder when importing into Locus without an internet connection.

thanks but this is really weird. It so simple file ... and even after import during online and offline, I see no problem. Hmm ... anyone some issues during import kmz or zip files with images?