Hi guys,
there were not too many changes in the Android itself during last years, that affect Locus Map. The big one was of course battery optimizations :o.
And now, tadá, we have another one as most of you know.
A11+ devices bring a limitation in access to the device file system. Task, which firstly was not looking so complicated, in the end, took more than a month of work for me and half of the Locus team.
I started this topic so I may answer and discuss any changes that the next update brings independently on the main app version topic(s).
----
Summary
- app can no longer access the root of the internal memory directly, only over very limited so-called "Scoped storage" access. This brings some limitations. Mainly fact that the neesd to move its main directory to the private storage (/android/data/...). Also no support for the custom "srtm" and "mapsVector" directory, no support for custom "geo tagged photos" directory, and no support for "external maps" defined over the app.
- app can't directly detect installed apps, so the content of the function panel may be limited
---
Michal is already writing a bigger blog post & manual for a better understanding of these changes and how to solve the most common use-cases.
You may watch the creation process of the main manual page in progress here (https://docs.locusmap.app/doku.php?id=manual:faq:android_11_filesystem).
Feel free to ask here, I'll gladly answer.
I'll also today finally publish the new Beta version where all these changes will be presented.
Hello menion,
thank you for your detailed explanation, although this will be devastating for many users.
A comment in the linked instructions is not always correct:
You have to move the maps to /Locus/Maps/ directory manually via the system file manager.
The file manager of my device "Samsung Xcover 4S" has no acces to Android/data ! :o
with e.g. "X-plore" or "Total Commander" i´m able to manage these folders
Really no access? Surprising. I'll inform Michal about some inconsistencies, thanks.
Import of the map will be implemented directly over the app next weeks. I had no time to do it yet. The solution is similar to the import of GPX etc. just app copy map into the proper directory.
And devastating ... well, as I work on it last weeks, I've ended with the (my) opinion that it is not a problem at all. I do all the necessary tasks I did before on my device and no problem. Only that the directory is not directly in the root of the device memory so it takes a few more clicks to find it ...
regarding "devastating" I mainly meant the memory consumption of the internal SD
Ah understand.
You may put the whole directory on the SD card! If you use a faster SD card, I believe it will be fast enough. But it needs to be tested, I've not tried it to be true ...
To be honest, I would rather not rely on SD card with its "ability" to handle database fast random read/writing, its stability etc. Not long ago I lost all my data on an SD card due to its malfunction and the necessity of formatting.
Hmm right. So maybe we should start recommend devices without SD card and bigger internal memory ...
I'm really shocked that none of the geo app providers joined forces and went up in arms to wake up Google.
The limitations are a disaster, as explained many times.
As far as I understand an existing Locus install (BEWARE OF UPDATES!) will continue to use full power of the existing app, right?
Quote from: Menion on November 26, 2021, 15:47:40
Hmm right. So maybe we should start recommend devices without SD card and bigger internal memory ...
And this is a BAD joke, Menion. In fact not a joke at all.
Quote from: Michal on November 26, 2021, 15:05:30
To be honest, I would rather not rely on SD card with its "ability" to handle database fast random read/writing, its stability etc. Not long ago I lost all my data on an SD card due to its malfunction and the necessity of formatting.
I appreciate your twofold warning, Michal!
Question to Menion:
I saw that many apps have a /Android/data folder on external SD, BUT their working directory in fact is on internal storage.
Are you able to access BOTH of them simultaneously? E.g. SRTM and large maps on external SD, while databases and other essential objects with high performance and reliability needs are kept on the internal /Android/data storage.
Just for my clarity:
Did I understand correctly that:
- both app and userdata may no longer be placed in the root directory.
- this means that userdata such as srtm and vector should no longer be in the root.
- this means that app and data must be in the private directory /android/data.
- that the app and data may be on the sd card, but only in the private directory.
Is it then possible to divide app and data over private directories in internal memory and SD card? For example the app with important data such as the database in the private folder in the internal memory and for example the srtm and vector folders in the private memory of the sdcard?
I personally like to have data (such as documents, books, photos, vector maps, etc.) together in such a way that a backup can easily be made and in my case all on the sd-card.
By the way, where could I find information about the devastating reasons behind this new setup in Android 11+. Maybe then I can understand. Years ago, I got rid of my Apple iPhone because I felt Apple was being too "protective" with regard to data I owned. Can I do what I want with it. I know; this is not the address where I should file my complaints. You can't do anything about it either. Fortunately, I know that you will do everything you can to keep things running smoothly. Thanks for that.
Sorry, I just saw that someone else has just beaten me to a similar question.
"no longer"
What about Android 10, 9, 8, ...?
I had a bad dream a few weeks ago where the app (beta?) surprised me with a notice that something had changed (I never changed anything!), I experimented with the "new normal", didn't like it, then changed it back. Did that really happen? Do I have to care about this if I don't have Android 11?
@michaelbechtold
we discussed this earlier and I have to repeat myself. Changes introduced now for internal memory were introduced 7 years ago for SD cards. They (these changes) complicate life to me, to our team, to many users also, but they make sense. There is also no serious reason why Google should grant an exception to the Locus Map app, sorry to say this.
And yes, version 4.4.3 is the last with the "full" access to the internal memory.
App has access to the private directory in internal memory and the SD card as well, so yes, this is an alternative solution to discuss > placing some data to private dir on the SD card.
@arnor
"both app and userdata may no longer be placed in the root directory" - correct
"this means that userdata such as srtm and vector should no longer be in the root" - correct
"this means that app and data must be in the private directory /android/data" - correct
"that the app and data may be on the sd card, but only in the private directory" - correct
"Is it then possible to divide app and data over private directories in internal memory and SD card" - not yet. If there will be bigger interest, I may work on it and this is definitely a doable task.
You may find a lot about it on the internet. The fact is that Google is step by step going the same restrictive way as Apple. As I do not know apple devices, I have no idea how far it goes now ...
@Viajero Perdido
There should be no changes on Android 10. If anyone notices any, it is definitely not an intent, so let me know.
Menion, you never really convinced re. below points! So, pls. ask Google via your DEV account those questions:
- how to share a DATABASE of 1GB+ between two compatible apps (they have no right to play LM4 XOR LMP - not their business, it's user's choice)
- how to share double digit data files (non-media!) between apps (like SRTM data or large map files (vector and raster); apps from different developers.
Looking forward to their answer.
Quote from: michaelbechtold on November 26, 2021, 21:43:06
between apps (like SRTM data or large map files (vector and raster); apps from different developers.
Just to clarify - so all vector maps have to be in an internal directory? Attachment related. Fortunately no problem for me, no sharing between apps.
Mbechthold, I wonder if gaining root and creating symlinks could be a solution.
Michael, ask yourself. The answer will be of course "Not everything is possible" :).
That's the same with Locus. Not everything is possible. Even an Android can't cover all use-cases by default. If you want more, you have to use the non-Google Play app or apps on rooted devices.
Tasker also offers a huge amount of options and many require a root of the device. Is a bad? I do not think so. There has to be a default secure setup for most of the users. And this now happens with this file-system change. Apps can no longer easily access the data of other apps.
Michael sorry, but I do not want to write to Google. It is a waste of time. You probably have a use case, where current changes cause a problem. I'm sorry for this, but you will have to find a solution on your own.
I'll work next year on some tools on how to detect wasted space in the app, unused maps, something with SRTM files etc. to reduce the required space of the app. And I believe, we will get used to it ...
Hello menion
at the beginning of the post you wrote
QuoteAlso no support for the custom "srtm" and "mapsVector" directory
I'll also today finally publish the new Beta version where all these changes will be presented.
I don't understand, I am using the new beta and my vector maps and strm data are still stored on the extSD and this data is used.
It's complicated :)
Google allowed one special and important exception.
In case
- you have installed a previous version that had full access, so 3.55.1 or 4.4.3
- and you UPDATE to new version with limited access
- system grant temporary access to old data, so the app can migrate data to a private directory
At the moment, you RE-INSTALL app (so you INSTALL app as the new one), this exception is removed and the app can no longer access internal memory directly.
You have a case we do not cover correctly. You already had data in the private directory but still working on custom directories. Seems there is missing some warning that these custom dirs stop work once ...
@ michaelbechthold.
I agree with you that it is a dirty trick of Google that you cannot even share (large) osm maps which do not contain any sensitive data to be protected. That app developers can't even stand up to the giant means that the fourth realm already exists. Docile and obediently follow what is imposed, no resistance allowed.
I migrated MapsVector to ///_extsd/Android/data/menion.android.locus/files/Locus/MapsVector
Unfortunately Themes are accessible, but not linked to the presets any more, probably presets use absolute paths? I see in presets-> themes then we also have to reconfigure all those theme specific details. That's unfortunate if you have many presets.
Maps and SRTM migration no problem. Fortunately no need for sharing them between apps for me.
Quote from: 0709 on November 27, 2021, 09:11:29
@ michaelbechthold.
I agree with you that it is a dirty trick of Google that you cannot even share (large) osm maps which do not contain any sensitive data to be protected. That app developers can't even stand up to the giant means that the fourth realm already exists. Docile and obediently follow what is imposed, no resistance allowed.
That is the sad thing indeed. No resistance. Not even trying. Even giants know fear. And they are able to learn, though slowly. And the populace has to re-learn to speak out and up (I am not endorsing silly Querdenker stuff here).
I read thoughts about rooting. Here things go completely wild. Google pretends to secure privacy. And their failure to think it through properly we are talking about rooting. THAT's a real joke :-)
Although I thank for the mention of linking. I thought about that as well last week. But I hoped that privileged apps like file managers might be able to establish such without rooting ...
Quote from: Menion on November 26, 2021, 23:25:08
Menion, that is not what your users can expect from you.
I am the most outspoken on that one - easy as a Locus user of many years and a comp.science person since 40 years.
But you should be scared of the silent mass who is not even aware of the pot. tsunami hitting them.
As a partial remedy you should establish the possibility to have both /Andoid/data locations used. Following Michal's point, the right approach would be to have the app function and performance critical parts on the internal private location, while the mostly readonly data like maps and SRTM and alike are allowed on external SD private folder as well.
Then capacity INSIDE the Locus only world will not bite - users do not have to buy a bigger phone :-)
My other challenges are not met, but for the mass of users above should be safe - and keep peace in the user base. And Asamm ...
Good luck.
Cheers
Michael
It's all too complicated for me. I don't even know what I have to do, what I'm allowed to do. I will only update my 2nd phone for now. With the other I wait until there is experience and especially a tutorial what to do.
Is there an address on google where I can complain. I do not want this security. I will create 100 accounts in case of need and write there.
After all, users will choose between internal/external SD. The precise place is something they not necessarily need to know. Locus UI could simply offer a matrix which let's the user pick the desired place with radio buttons (so, pick int or ext sd) for vector maps, srtm, whatever else... And it should manage the file movements in a secure way.
I don't like the move towards apple, where people even don't need to be aware of files and folders. But otoh, bullshit apps, malware, messing with other apps, idk, most people have just a glimpse of clue and need to be protected 😐
Quote from: freischneider on November 27, 2021, 12:12:57
It's all too complicated for me. I don't even know what I have to do,
This is what's expected from you: Put all configurable stuff (by Locus, ie currently SRTM, Backup and MapsVector) to their traditional places below:
/<int_sd>/Android/data/menion.android.locus/files/Locus/
OR
/<ext_sd>/Android/data/menion.android.locus/files/Locus/
AND keep in mind a) data get deleted on uninstall of Locus b) data cannot be shared with other Apps.
That's the future 😊 See my screenshot.
But as long as your device is on Android 10 or less imo there's no need for you to change paths.
a) can be really ugly, eg here maps and srtm are 50+ GB.
Terminology is really confusing here. "Internal storage" vs "Internal memory" in the Set Working Directory dialog. "Internal" vs "External" in this discussion, but both seem to be inside the phone somewhere. Those who know that one is removable and called "SD" would be confused by an "Internal" option being /sdcard/Locus, which (I think) is in non-removable memory despite the "sdcard" name. I think only an Android developer could understand all this.
Here's the chance to be clear and non-confusing. And if there's no need to worry because you're on Android 10 or lower, then please say so in the UI. Thank you.
I would love it if ALL path configuration would be on one options page, with a clear terminology (eg "Internal storage / removable storage"), with the paths visible (currently they are hidden behind question marks), but clearly just informational, with information on free space, used space.
If there's no removable storage, nothing can be changed (?), then the page is just informational.
So exports can only saved to the private directories in Android 11?
@tapio
hmm, there will be probably more places where the app uses absolute paths. This is a little unfortunate as I never expected that the app will have to move its private directory to a different location. Sorry for this ...
"Exports" .. right, currently it is not possible, but it is doable. Till now, it was not necessary, now it is, understand.
@freischneider
The best solution ... ignore this topic. If you do not use anything special, the app should just offer "migrate data" process after the update, that was what we tuned all these weeks before. So hopefully, it won't be complicated.
---
Terminology problems ... hmm probably we missed something, thanks guys, we will look at it with Michal.
Quote from: Menion on November 26, 2021, 19:28:21
"Is it then possible to divide app and data over private directories in internal memory and SD card" - not yet. If there will be bigger interest, I may work on it and this is definitely a doable task.
1000% supported! I have very big non-vector maps (OruxMaps format) on extSd, which won't fit on intSd, but also don't want to take the performance hit to database operations on extSd.
I don't want to spend lifetime getting angry at Google for this change, we few people won't be able to change it. And I don't really care about the actual paths to data (I would care about inability to share this data, but I don't need that, so I'm fine). But if there's any way to make use of the data on int and ext SD at the same time, that would be a big improvement! I would probably wait with updating Locus if you now tell us, you'll tackle this... Thx for considering!
I also stumbled over one sentence in the docs (under "What if I refuse"):
"you can load the original data any time from the app main menu or from settings > miscellaneous > default directory"
I honestly don't understand that - what does "load" mean in this context? Do you mean I can start the copying of the old (original) data at a later time? However, the sentence more sounds like "you can start Locus without copying the data and then change the default directory at runtime and have access to the old data again".
Anyway, this should be worded more clearly.
@slarti76
I'll rather not promise that this feature will be implemented. A nice help desk topic with a huge amount of votes may convince me ;). Because it will mean quite a lot of work to do it properly. The priority now is perfectly working migration to new app version & migration between Locus Map Pro and Locus Map 4. So currently there is no time for this kind of stuff.
---
Terminology united across the app, thanks. On all places will be used "storage".
Menion,
I am not sure you understand the risk Locus and Asamm are in.
Pointing to voting is denying intelligence. Why?
Take the max voted topic, and divide it by the number of users you have. How many zeroes behind the dot - before a non-zero digit appears?
In other words: your risk is the unknown, and the invisible. And pointing to a hence meaningless voting is slapping those unseen > 1000000 users in the face.
Now, let's employ some intelligence. Proposing people buy new phone with more capacity - even as a joke - is not. Do you agree?
Another fact is, that most Androids are NOT Pixels. But rather the majority employs external SD card support, to allow for cheaper storage extension. For mass storage eaters like maps, e.g.
Then take Michal's advise (no DBs on SD, please). Put everything together, and employ your own proven intellect.
Quote from: michaelbechtold on November 29, 2021, 20:45:37
Menion,
[snip]
I really have to agree with Michael here. I have no problem to open a Help Desc topic, that's not the point. But this is too slow for this important topic. I have friends for whom I am the "Locus guru", but who would never take the time to log in on the help desk or whatever. And they all have big maps on extSd that they can't move to internal.
And I'm still not even sure from the Manual entry: Does Locus even offer to move all data to extSd instead of intSd on first start with new version? The manual is not explicit on this. I'm honestly not willing to try out Beta this time (even though I always do), because this is obviously a one way street. I would be ok if you say "Google forces my hand, for now you'll just have to do it, and you can move to extSd, but next I will implement way to divide big and small data between intSd and extSd". But making this dependent on a lengthy voting process is very disappointing - and potentially p***ing off many users. As Michael says, don't underestimate the amount of users with cheap phones with small internal SD, slapping a big MicroSD into the slot...
PS: plus over time the track DB alone gets HUGE. So you have quite some data for internal storage to keep performance alove.
Mine has crossed 1 GB last month. And there are people who are more active than I am.
I missed my bet for injury and stay in the hospital too. However, I agree that when you install a new version, it should be possible to determine whether the maps to EXTSD. For example, I will have LOCUS with And11 in Samsung A8 tablet. I am sure that all maps would not fit in the internal memory. It is also important that private folders are also accessible after installation for file manager such as Total Commander. Otherwise, you will not be able to record foreign sources.
CZ: Betu jsem kvůli úrazu a pobytu v nemocnici taky nevyzkoušel. Souhlasím ale s tím, že při instalaci nové verze by mělo být možné určit, zda mapy uložit na extSD. Já například budu mít Locus s AND11 v tabletu Samsung A8. Jsem si jist, že všechny mapy by se do interní paměti nevešly. Důležité taky je, aby soukromé složky byly po instalaci přístupné i pro správce souborů jako třeba Total Commander. Jinak do nich nebude možné nahrát mapy z cizích zdrojů.
@michaelbechtoldwe all are well aware this is the problem (Android 11+ limitations). On the second side, to be true, I'm still not convinced, this may be a major break in the app usability. I'm looking forward to next week when we finally find the correct answer.
Topic & voting: I've expected, it may be a task that should take quite a lot of time, that I currently do not have. So voting may help me better decide if this isn't the idea of just a few people here on the forum ...
Quote from: michaelbechtold on November 29, 2021, 20:45:37
Another fact is, that most Androids are NOT Pixels. But rather the majority employs external SD card support, to allow for cheaper storage extension.
This is based on known facts? We are talking about devices with Android 11. Most of the modern devices are created with bigger internal memory instead of support for external SD cards. Sure, SD cards are usually 4x time cheaper, I know. Also, I made just a quick benchmark, and LoMaps from the SD card (average class 10 card) is 9x slower than LoMaps from the internal storage. You are the one who fights for faster maps ...
@slarti76Manual updated a lot, hope for good.
@Žajdlík JosefUnderstand. To have fast maps, it will be necessary to limit number of maps on the device. As I wrote above, SD card, no matter what I do, will never be too fast.
New version
MapGooglePlay_4.4.3.31 RC_1045 just published.
It should have mainly
- option to move mapsVector and SRTM directories to the private directory on the external storage (SD card)
- integrated links to manual pages
Let me please know mainly if this nighly update in custom directories will work correctly for all of you. Thanks
Menion, I have tons of large maps on SD card, even on older devices. There is no performance problem using them in higher ZLs (although for sure you can MEASURE a significant difference).
A DATABASE on SD however, I would not even try, because the effect is obvious.
To be true, I expect the opposite. Working with points/tracks is usually one time ... a load of a bunch of points, save the single track, hide a few points etc. On the second side, maps are loading every time you work with the app ... anyway I fully support database data on the internal memory, mainly because of possible data loss due to malfunction of SD card.
@Menion: you write so that it is important with the cards so that they are internal, because the internal memory is faster. I also thought so and have them internally. I also have the database internally.
But now there is little space and I have the SRTM elevation data on the external SD card. Have there also much LIDAR (need more space).
Question: How important is the speed for the elevation data ?
Translated with www.DeepL.com/Translator (free version)
Hello menion
Quoteoption to move mapsVector and SRTM directories to the private directory on the external storage (SD card)
to check the function i moved my srtm data from the custom folder on the extSD to the private folder of the extSD with the locus filemanager.
There was no error message and the elevation data is also displayed on the map.
Unfortunately, none of my file managers show me the contents on the extSD/Android/data.
Downloading missing elevation data from another map area also works, but these are not displayed either.
So I have no way of managing these files.
hi Wolfgang, I've tested many file managers and Solid explorer ans X-plore should tackle this. But on Xiaomi phones, none of the managers succeeded in transferring the files manually. Only connecting the phone to a PC managed to do the task via Total Commander.
Odesláno z mého M2101K6G pomocí Tapatalk
Hi Michal
i´m on Samsung Xcover4s with X-plore ???
OMG, Total Commander dev. is struggling and fighting currently. I am currently unclear about how I could copy OAM maps to Ext/Android/Data/<locus>
https://ghisler.ch/board/viewtopic.php?t=74859&start=15
@tapio
exactly this is what I've discovered a few days ago. Access to int/Android/data may be obtained by the small hack and Locus Map uses it to transfer data between Pro and 4 versions. Seems to be a security hole in Android to be true.
On SD card, this hack does not work and there is no known method how to access app private data stored on the SD card as I know.
This also answers @balloni55 post above.
Every app which operates with external data (like Locus vs. OAM maps eg) - What is Googles idea here? Only the app itself is allowed to retrieve/store such data? This would be a huge step back (and a huge step towards Apple stupidity).
Same country, same ideology, same stupidity.
Instead of the desire to install the latest Android UPGRADE more important becomes to be able to install a still functional flexible Android version by DOWNGRADING. You have to worry more and more about trying to upgrade or not.
Guys, inhale, exhale, repeat...
You are right, Tapio. But a lot of flexibility is lost, isn't it? It is no longer very pleasant.
Quote from: 0709 on December 02, 2021, 16:27:13
You are right, Tapio. But a lot of flexibility is lost, isn't it? It is no longer very pleasant.
If I have to decide between performance and still-being-able-to-use-all-my-maps, that's much more than "flexibility". For the first time since I'm active in Locus testing-and-giving-feedback, this could be a make-or-break-moment for me. My outdoor phone (XCover Pro) is neither the fastest nor has the most internal memory. I'm not gonna use my "good" phone for outdoor stuff, so I'm not gonna say it's a no-go, will have to see how it performs, but I'm really anxious...
Hello menion
question,
if a user moves his Vector maps to the extSD to Android/data and later wants to add e.g. OAM V5 map with the corresponding theme, how can this be done if the folder structure is not visible to the user?
I just tried to populate the private mapsVector on private external SD and failed with both, Solid Explorer and x-plore.
You can READ private data on external SD, but I failed writing (copy some file in a test)
That would leave two options:
- take SD out and put it into a PC/MAC/linux box and do the dirty work there
- use adb while connected to one of above
I have done the first, but only read about the second.
@Menion:
I could not wait until next week because I am so curious ...
So I started trying, not on my S10 for daily use, but on the Tab S6 light (64 GB total internal space, large external SD) and found some essential problems.
My plan was to use the private section of internal SD (/Android/data/menion/...) as the main Locus location, then set the external SD for mapsVector and SRTM.
What happens?
1 long start - question: do or postpone
2 offers move
3 LMP loss of access warning
4 selection private - long pause 3 GB free - not enough - cancel
See attached for the situation: need for 1 GB (see external SD option), 3 GB free on internal SD.
Why is that? Most probably because of the SRTM and mapsVector that sit on external SD and require 75 GB in total. So I should have unchecked the two choices, right?
Which brings me to an overall judgement:
1) the steps and explainations are quite OK, but
2) it lacks overview for the user and
3) a plan
And there is a good chance that a user has ONE view of the storage reality, while Locus has another. Recipe for disaster, pain and discussions.
Hence I propose you do not offer the migration immediately, but produce a full storage picture how Locus sees it.
All the elements, with their sizes, and the options how and where to move them. And send this as a mail (share ...)
Something like:
Current Locus storage layout
=======================
Locus main: internalSD/Locus 1 .5 GB
Locus mapsVector: externalSD/LocusMaps/mapsVector 20 GB
Locus mapsVector: externalSD/LocusMaps/SRTM 55 GB
Internal SD capacity total/free: 64 GB / 3 GB
External SD capacity total/free: 256 GB / 151 GB
Next: what should a non-IT person do with a non-trivial situation?
1) understand the picture
2) understand the options
Can people derive that themselves? Hardly, hence they need advise.
A text that explains the options given various scenarios should be added to above report:
1) the total Locus storage fits fully into internalSD: do whatever you want, but internal preferred for data safety and performance reasons
2) current data on externalSD does not fit into internalSD: ...
3) ...
some homework here for all potential cases.
My setup above would need the following actions:
1) /Locus goes to internalSD
2) AT THE SAME TIME (!!!) current SRTM and mapsVector need a choice for their final target (I know you could uncheck both and doe manual stuff later, but that is not good, because it leaves a number of people stranded, in particular as the file managers fail so far)
Also: give warnings about long wait times for steps that take long, and tell users to click "Wait", not abort. Some of the steps 1-4 take unexpectedly long. I trust for a good reason, but people may wonder ...
So far my 2c
Cheers
Michael
Just want to add that I also experienced long waiting time, like 15 seconds. See screenshot.
Definitely long times, hence my request to put a warning into the documentation.
But things get worse than that:
- also connection to linux via file system integration (Nautilus and alike) stop at the /Android/data/.../Locus folder, which is shown empty, although it is not.
- trying to migrate a huge SRTM folder makes Locus die after quite some time.
Has someone tried to use adb file system access to those private folders?
If the latter does not fly, dire times are ahead.
As long as Locus does not provide 1" resolution DEM files, there will be no way - that I know - to make Locus work with better DEMs than Locus download provides.
Also it would be bye-bye to side loading other map files than LoMaps. You could only use map sources that provide Locus-Actions to install with the help of Locus itself.
For 1"-SRTM Menion would need to provide some download feature for Sonny's DEM data e.g. With wild card selection.
Someone has smarter ideas to overcome those limitations?
As of now (late 2021-12-02) today's findings mean: stay away from the latest Locus version for now.
And NEVER EVER deinstall the current version.
Update: Total Commander allows WRITING into the INTERNAL SD private folder (hence more powerful than Solid Explorer or x-plore - interesting).
However: this is was the TC wizzard also posts:
ghisler(Author) » 2021-06-20, 06:33 UTC
Unfortunately as I already wrote:
Note that Android/data on the external SD card cannot be accessed with any method.
I have tried the tricks to pass the correct path to the Android file picker, but it doesn't want to go there. So even the Android files app cannot access this folder. The only way to access it would be to "eject" the card via settings, take it out of the phone, put it in a card reader, and then access it from a PC or via OTG adapter from the phone.
We need someone who can beat the sh.. out of these Google so called Android architects. Narrow minded morons they are.
Update: Test from PC
From a W10 PC I could even write to external SD private folders. Puuuuhhhh.
That allows some room to maneuver for quite a number of people.
Still a shame on Google's part. Arrogant, not thinking, not listening, not trustworthy on the technical side either.
Did anybody try things with a rooted phone? I assume there every directory should be accessible?!
@michaelbechthold
QuoteUpdate: Total Commander allows WRITING into the INTERNAL SD private folder (hence more powerful than Solid Explorer or x-plore - interesting).
for me it´s possible to write also with X-plore to INTERNAL SD private folder
TXs, Wolfgan. Was late last night, and I mistook external for internal.
So, while this is nice, it does not help offloading large non-vector maps to external.
@Menion: can you PLEASE do the same thing that you did for mapsVector also for maps?
Also it seems that you look for mapsVector on internal although it is configured to external SD.
So, writing to the configured location, but always reading from both locations is the most powerful approach. And easy to implement ...
Thank you.
App Data Transfer: there are two entries with the same title doing opposite things. Not good.
But how on earth can Locus WRITE to the root folder? Is that the grandfathering of an existing installation?
Would this go away after a re-install?
@balloni55
Adding OAM is easy thanks to integrated direct download from OAM to Locus. Just use the method that directly downloads maps from the web, not manually.
@michaelbechtold
As I wrote before:
File-browsers with "External storage" permissions can usually- read/write to all directories in the root of internal/external memory
- read/write to all app private directories (Android/data/) in the internal memory thanks to the small known hack (that I believe will be patched in one of the coming Android versions). So not all filebrowsers are able to do this!
- read (maybe), but 100% no write to all app private directories (Android/data/) in the external storage.
There is no known method that is able to bypass Android integrated security for now.
From our internal testing, you may easily connect the phone with a cable to the PC and copy data manually over the PC! No need to ADB!!
Thanks for the detailed long post. We will think about possible improvements.
Slowness you and others noticed will be most probably caused by the computing of "Used disk space". With the new Google system this tiny operation took really loooong. Unfortunately. I'll look at it.
Quote from: michaelbechtold on December 03, 2021, 09:30:04
App Data Transfer: there are two entries with the same title doing opposite things. Not good.
Where?
Quote from: michaelbechtold on December 03, 2021, 09:30:04
But how on earth can Locus WRITE to the root folder? Is that the grandfathering of an existing installation?
Would this go away after a re-install?
I'm a little out of context here, sorry. A better explanation is needed.
TXs Menion.
File browser thing clear, and also seen it in practise now.
PC connect to Windows is also easy as you write and successfully tested.
The last two topics relate to settings - maps - standard folders - file manager.
After migration to private internal two new entries appeared under "App Data Transfer"
There is the naming problem.
And the option of Locus writing to root folder. Grandfathering of existing install?
The naming duplicate is in German.
My English Locus install on S10 I will not touch for now.
English is OK. Translation error ...
Quote from: Menion on December 03, 2021, 10:28:25
- read (maybe), but 100% no write to all app private directories (Android/data/) in the external storage.
There is no known method that is able to bypass Android integrated security for now.
I have a rooted LOS 18.1 phone and this (most unknown) path helped almost every time with every app:
See attachment
"root" ... oki :)
This topic is partially covered in the new version topic, but it still has its purpose > focus on the file-system. So cross-post to new Beta: https://forum.locusmap.eu/index.php?topic=7632.msg65883#msg65883 ... if you find a moment, give it a try and test new "maps" and "mapsOnline" setup if it works as expected. Thanks guys!
WHERE IS MY MAPS ON internal sd ??? GONE!
To the external SD structure. Worked like a charm. The online thing I want to keep internal.
Migration of maps and mapsonline worked flawlessly. As mentioned, percentage indicator is not optimal if it only changes once per file.
@michaelbechtold
you lost some maps or how should I understand your post?
@Menion: you write so that it is important with the cards so that they are internal, because the internal memory is faster. I also thought so and have them internally. I also have the database internally.
But now there is little space and I have the SRTM elevation data on the external SD card. Have there also much LIDAR (need more space).
Question: How important is the speed for the elevation data ?
Quote from: Menion on December 05, 2021, 12:50:56
@michaelbechtold
you lost some maps or how should I understand your post?
That was a happy fun message, documenting success. They were gone from the old location /1st line) and ended up at the target (2nd) sentence.
Sorry for the confusion.
Summarizing the experience and discussions from the last weeks:
- the current support for SRTM, mapsVector, maps for both, INT and EXT SD private space is powerful enough to help all use cases, i.e. map storage needs, capacities etc. - well done! A big THANK YOU!
- the backup to ANY folder, even outside private space is great - although I do not understand how you could escape the Android 11+/API 30 limitations. Well, these are not files with need for random access (like DBs and maps); maybe that helped. Will it survive Google "fixes" (below the level of yet another storage model) ?
- the GPX export to the backup location would be really nice
- guiding the community of users who are not able to help themselves to make the right choices is the gap I see
Fingers crossed ...
Cheers
Michael
Quote from: freischneider on December 06, 2021, 10:20:44Question: How important is the speed for the elevation data ?
I'm using them on ext sd quite a while and noticed no problem at all.
Even on the migrated Tablet the SRTMs are in their original place. EXT SD, but not in the private structure,
Menion might be able to tell pot. difference in performance.
@freischneider
SRTM data needs fast access when you use hill-shading. Anyway, truth is that HGT files that are currently used should be cached in memory, so it should be slow only for this first-time app to use it. I'll have to do some benchmarks, but if you do not feel any difference when using hill-shading, no worry, and use it directly from the SD card.
@michaelbechtold
thanks for the summary and help with this complicated topic! The current state is definitely not a final version. We will be updating also based on the problems and support on the help desk!
Difference in performance > as mentioned in second topic - it will need some benchmark to verify the real impact on battery and performance.
I am not sure. I think, after moving "maps" to the external storage the map in "Preload global map" wasn't configured any more.
Here I zoom from the global map into some random area, which is not cached. Hillshading is on and I use Sonnys 24mb hgt files. SRTM on ext sd (those are "NM" cards on Huawei). Map is OAM V5, Device is Mate20X.
I don't think it was faster on int sd. Definitely alright for me.
https://youtu.be/e0dnk8q71Gk
Same global map thing in my migration. But easy fix to set it right manually.
Yes, there may be a few more settings lost (like a global default world map) when moving the map. I never expected that something like this will happen so on a few places in the app are used absolute paths. Sorry ...
Quote from: michaelbechtold on December 07, 2021, 12:54:24Same global map thing in my migration. But easy fix to set it right manually.
Of course, just thinking about newbies or people being unaware, it's after all a setting that you usually set and forget...
Quote from: tapio on December 07, 2021, 16:42:36
...
Of course, just thinking about newbies or people being unaware, it's after all a setting that you usually set and forget...
True.
I have not tried a migration with totally different root level map folders in place.
Something like /ExtSD/OAM or /IntSD/World, outside /Locus.
Will Locus migrate them all? Automatically?
Somebody has experience?
For sure @Menion can tell :-)
@Menion, please take note! Message from C. Ghisler, tcmd dev.
https://ghisler.ch/board/viewtopic.php?p=407841#p407841
QuoteTry asking the author to use Android/media on the SD card instead of Android/data. The app should be able to access this folder without special rights, and other apps can write there just fine.
Quote from: tapio on December 12, 2021, 16:18:24
@Menion, please take note! Message from C. Ghisler, tcmd dev.
https://ghisler.ch/board/viewtopic.php?p=407841#p407841
QuoteTry asking the author to use Android/media on the SD card instead of Android/data. The app should be able to access this folder without special rights, and other apps can write there just fine.
Maps in the media folder - that would be awesome. Well, not awesome compared to pre-A11 behaviour - one might ask what's the point for Google to enforce all these changes, when there's still a folder that can be accessed freely? -, but still. If it works, I'll take it!
Hi Android 11 users.
Can you have a small test for me by Android 11 pse ? Someone ?
Download open Andromap by a browser ( Ex. Andorra).
For the next actions (I) do use the Cx Explorer (recent updated) (or any other capable explorer)
By Cx explorere unzip the Andorra.zip into:
Folder Downloads > folder mapsVector > folder Andorra > = Andorra by 3 files
Also you can dowload a theme. Do not unzip !
Put the zip in Downloads > (example in) the Folder mapsVector.
Install the Cruiser app.
https://play.google.com/store/apps/details?id=gr.talent.cruiser&hl=nl&gl=US
Open the Cruiser app:
By the Cruiser hamburger menu:
Select map > open maps > open maps (or add maps) > By Cx Exlorer (or other) > set Downloads > mapsVector > Andorra > Andorrra_oam.osm.map > set.
The Andorra map should show up now. Map displays correctly ?
By the same Cruiser map menu you can select a standard map style
OR
Install a theme this by the same procedures by using Cx Explorer. (Do not unzip but directly slect theme.zip !)
In the Cruiser map menu you can set the thema map layers. Display correctly ?
I do know that Emux the dev from Cruiser and Kurviger app had a change about a year ago in order to prepare the transition to A11 where he replaced the app file explorer than by an external file explorer. (helper) (Emux recommanded than to use Google Files for these actions)
https://forum.kurviger.de/t/app-storage-access-framework/4409/20?u=0709
https://forum.kurviger.de/t/app-storage-access-framework/4409/28?u=0709
Hmm... I'm experiencing very slow performance access from lm4 after relocating all my maps/themes etc from their ext_sd root folder you
To their own lm4 ext_sd private folder. Lm4 takes along time to load now with a blank screen for 15-25secs, opening the themes menu is slower and drawing the maps is significantly slower, to point that I would consider it almost unusable.
I kept a copy of the files and restored them after the move, since lm3 is still using them in the root directory. Lm3 is still running fine and snappy, no delays loading or using themes menu.
I was going to uninstall lm3, since I don't want two copies of my maps, and lm4 was looking OK, until now. Now I find using lm3 more of a joy...
Is there a known issue for this performance hit? I don't want to put all my maps on internal memory as they are over 32gb.
Maybe a really dirtry trick could help - 50:50 chance; I'd guess:
I found that when you move maps/Vector to extSD, LM 4 still screens intSD private mapsVector.
x-plore, Solid explorer and TC can work wizth intSD (on many, maybe not all) devices.
So you may copy the theme folder to intSD. I found x-plore can delete folders on extSD. Hopefully, after a deletion of the themes from extSD Locus would use the ones found on intSD mapsVector.
The mapsVector SETIING however should stay with extSD inside Locus, becasue there is no other way to get maps to extSD...
Good luck
Cheers
Michael
PS: I found the same pain on my Tab S6 lite (also reported it), but have not found time to try above (because it only came to my mind when I read your report :-)
Well I tried Michael's suggestions:
- moving the themes to IntSD in private folder (while custom folders still point to ExtSD, did not work. Even worse it took ~120 secs with a blank screen on loading, and there was only the internal theme available.
- I notice there were an empty set of Locus folders in IntSD\Locus\ etc, tried moving them there, say result
- Lastly I moved them back to the ExSD\android\data\men... folder and then it returned to the slow loading of about 20secs (~10-15 of blank screen) and still very slow to use...
So.. still looking for a solution/improvement.
I really don;t understand why LM4 was working fine using everything on ExtSD before the Android 11 relocation thing, and now is really sluggish? I sometimes get a pause and the android OS kicks in and asks if I want to close or wait for locus map... never did this before moving the data to private folder...
I understood from earlier posts that the access to EXT SD content is less efficient and the order API calls are no longer available. I saw that too, when installing themes on ext SD via Locus Actions from OAM. Hence my dirty idea. Pity it did not work.
But it would be up to Menion to always look at int SD mapsVector for themes, even if the maps are on ext SD.
@Menion: what is your judgement, pls. ?
Quote from: michaelbechtold on December 14, 2021, 11:49:42But it would be up to Menion to always look at int SD mapsVector for themes, even if the maps are on ext SD.
@Menion: what is your judgement, pls. ?
Sounds reasonable, and I additionally think the theme folder name is just a historical, but ugly thing (mapsVector\_themes). I'd even drag them up one level, so that we may have all extra content on one level...
Maps\
MapsOnline\
MapsVector\
MapsVectorThemes\
SRTM\
..
Quote from: tapio on December 14, 2021, 13:06:23
Quote from: michaelbechtold on December 14, 2021, 11:49:42But it would be up to Menion to always look at int SD mapsVector for themes, even if the maps are on ext SD.
@Menion: what is your judgement, pls. ?
Sounds reasonable, and I additionally think the theme folder name is just a historical, but ugly thing (mapsVector\_themes). I'd even drag them up one level, so that we may have all extra content on one level...
Maps\
MapsOnline\
MapsVector\
MapsVectorThemes\
SRTM\
..
+1
It's always a pain to set up syncs (e.g. with FolderSync) that sync the themes but not the (big) map files.
Quote from: michaelbechtold on December 14, 2021, 11:49:42
I understood from earlier posts that the access to EXT SD content is less efficient
What I do not understand is that I haven't changed anything. Locus Map was working with everything located in \ExtSD\Locus\mapsVector (maps and themes) and all that has changed is that now all those files are located in \ExtSD\Android\data\menion.android.locus\files\Locus\mapsVector.
Why should moving and using the exact same files from one location in ExtSD to another make such a difference to the performance? LM3 is still using the \ExtSD\Locus folder and performing as expected. I've seen others complain about the performed on LM3 after the relocation of files...
I guess I can't move the files back to \ExtSD\Locus since this is now marked as not allowed in the app... besides I will eventually hit a problem when I need to add new maps or uninstall/re-install the app...
From Android 11 on (API 3ß+) it has been GOOGLE breaking working solutions by their silly and shortsighted changes.
Hi guys, give me please some time till the end of the week. I'll then answer all questions. Thanks for understanding.
So I managed to move the maps to the private folder both on my Android 11+ phone and tablet, but on the tablet (Samsung A7) I had to do it via my Windows laptop. None of the three suggested third party file managers can do the job completely. X-plore comes the furthest - it can open all of the folders and view the content, it can even create new folders within them, but can't copy the file contents :-\
My question is - does anybody know of some file manager that actually has full control of private folders on Samsung A7 tablet? Or some other Samsung tablet, I suppose they all work similarly. It would really be helpful if I had full control over the files, without the need to resort to an external device for it.
There is a difference between int and ext SD private folders.
Google did not only create this mess by they new policy, they also failed to implement the MANAGE_EXTERNAL_STORAGE in a consistent way.
Hence even x-plore which has this permission cannot write files to ext SD private folders, but it can create folders ...
PC USB connect is the only way so far.
If it wasn't so braindead, it would even be funny that Google allows app X to access internal data files of app Y on intSd, which cannot be accessed without Android, while disallowing it on an extSd, which you can always just yank out and attach to a Windows machine where due to exFAT you can do whatever you like wherever you like...
Well, this access required this strongly regulated permission.
To me the braindead thing is to forbid access to root level folders. For sensitive stuff they could and should restrict access to private folders. And app developers should use these places for sensitive stuff.
But something like /Locus was never private in the sense of secrets or security challenge.
Quote from: michaelbechtold on December 16, 2021, 12:57:40
Well, this access required this strongly regulated permission.
To me the braindead thing is to forbid access to root level folders. For sensitive stuff they could and should restrict access to private folders. And app developers should use these places for sensitive stuff.
But something like /Locus was never private in the sense of secrets or security challenge.
Maybe you can request an exception from google for less sensitive data. I would write to google. But I do not know a contact person.
Only Asamm could do that for Locus, but they said Google is not responsive or supportive, so they did not try.
Hopefully the approach via /Android/media will work and be performant on both, int and ext SD. Then, with some other smaller changes the pain and turmoil should be over.
Fingers crossed!
Summary of the current state ...
So after a push and push on the Google support
- Locus Map 4 passed a request on "All files access" (don't ask me how this is possible, probably some India guy had a bad day). So the latest app on Google Play has temporary!! exception. We have a little more time to prepare better on limited access now.
- Locus Classic (Locus Map Pro) request was declined. Currently we are waiting on another attempt, but as expected, this should not be allowed to such kind of app at all
- latest Locus Map 4 already has enabled support for /sdcard/Android/media/menion.android.locus/maps(Vector) directory, so feel free to try it. Just insert maps there, the app should load them automagically during (re)start.
Feel free to ask if something is not clear.
Quote from: Menion on December 21, 2021, 10:08:18latest Locus Map 4 already has enabled support for /sdcard/Android/media/menion.android.locus/maps(Vector) directory, so feel free to try it. Just insert maps there, the app should load them automagically during (re)start.
Thanks. "Some indian guy" - hrhrhr.
Please (if not already happened):
a) add support for the media folder in the UI (prefer it over Android/data)
b) support Android/media on Android <11
So all users use the same places, makes support easier. I'm on Android 10, but I want to migrate all my stuff to future-safe places.
Question: AFAIK if files are in the private folder under Android/data, they are deleted when uninstalling Locus Maps, we all know this is a big deal if we have tons of Maps and SRTM files there.
Does this also apply to Android/media?
Quote from: Menion on December 21, 2021, 10:08:18
...
but as expected, this should not be allowed to such kind of app at all
...
Given the unprofessional carelessness of the Google architects, I can agree only in principle, but not in practical terms. They simply forgot use cases beyond pictures and videos, and ignored hard performance requirements when it comes to SD.
Hence I consider it just adequate that as a temporary remedy they grant such permission.
And if /Android/media show prove to be the magic wand (fast in any case, no delete at uninstall), then they should have promoted that as "best practise" for above use cases.
Fingers crossed ...
Hi,
how can I force Locus 4 and Locus classic to use android\media\locus...?
Sorry I had always rooted smartphones (now LOS 18.1 Android 11) and without these new problems I did not follow these threads.
Bot now I have a new Sony Xperia 5 ii (Android 12) and for the next time without root :-(
Hello, "Android/media/x" directories are automatically used for maps. So for example in the case of Locus Map 4, maps placed into "Android/media/menion.android.locus/maps" are automagically used ...
@menion
thank you, I will try it.