Locus Map - cloud/sync server

Started by Menion, September 08, 2020, 13:06:05

0 Members and 1 Guest are viewing this topic.

Menion

It's almost 9 months since we started the work on a cloud/sync solution (since Jan 2020).

And today, we are in a moment that all internal tests were successful and the first public test is possible. Locus Map team currently consists of seven people, but still, a lot of corner cases may not be covered. Because of this, any public tests are more than welcome.

Important: use on own risk!

Important: make a full backup of your database!! This version modifies the database upon start and also may cause a data loss in case of any serious problem in client-server communication.

Version: Google Drive

Privacy: we do not yet have complete "privacy policy" document. Anyway, be aware that based on GDPR regulations, we are trying to work with data as securely as possible. We never share, re-sell etc. any data, never. It's against our deep conviction (I personally guarantee). Once Locus Map 4 will be officially published, all uploaded data on the server (not on your device!) will be deleted.

Usage:
- by default, sync (synchronization between client-server) is disabled
- the best start-point is currently over Data Manager -> in the toolbar is the "Sync" button (will be improved later)
- after the first full sync is done, automatic-sync is enabled. After any change in the database, the sync will be done automatically (with 30 seconds delay). This may be disabled in sync-settings (to work fully manually if needed).

Reporting problems
As usually, best is reproduction of problem on my own device. This means:
- a sharing your original database backup + steps to reproduce problem
- or precise description that may lead to this problem
- or if the app crash, creating a bug report by well-known method here

Known missing functionality & bugs
- automatic notification on other users devices to download new changes

Web route planner & manager of own points/tracks is in preparation (the second part of sync-system). ETA for the Beta test of this part is currently 2-3 months. For now, this tool is mainly about "cloud data backup" and "syncing between own devices".

Looking forward to some feedback ;)
- Official help (ideas, questions, problems): help.locusmap.eu
- Advanced topics, sharing of knowledges: you're here!
- LM 4 Beta download, LM 4 Release download

balloni55

Hello menion
thanks for neu sync version
My first experiences
- in portrait view not all text is visible
- the error text should show the reason for the error
- after enabeled wifi syc process finished but i get an error report
- if i now click the sync button again locus close
LMC 3.70.0 AFA
Locus Map 4.22.2.1 Gold AFA
LMC User ID c8b19276f
LM4 User ID e06d572d4
  •  

Menion

You are fast, thanks for tests!

- cut texts ... hmm, even four lines are not enough. Oki, I'll improve it
- error when wi-fi is not enabled: ah, good point, thanks!
- the last one will be the worst. Error "-1" should never happen, it is something like "well, I really do not know what happens". If the app always crashes, should be possible to create a log for me? Thanks!
- Official help (ideas, questions, problems): help.locusmap.eu
- Advanced topics, sharing of knowledges: you're here!
- LM 4 Beta download, LM 4 Release download
  •  

Tapio

#3
OK I have my own backup strategy and will not upload my database - in my case my whole movements - unencrypted to a 3rd party. Would only do that if encryption/decryption was transparent and client side.
But I'd participate in testing if I could pick specific folders in my database for syncing - is that possible?
Tapiola MFV4+ theme for OAM Maps:
Discuss - Releases - DL latest - Install latest
  •  

Menion

This is not a problem. Whole "sync" system is 100% optional feature, not a requirement.

Selection of groups to sync is currently not possible. This feature is in my "ideas to consideration" list for version 2.
- Official help (ideas, questions, problems): help.locusmap.eu
- Advanced topics, sharing of knowledges: you're here!
- LM 4 Beta download, LM 4 Release download
  •  
    The following users thanked this post: Tapio

balloni55

Quoteshould be possible to create a log for me? Thanks!
log send just now
LMC 3.70.0 AFA
Locus Map 4.22.2.1 Gold AFA
LMC User ID c8b19276f
LM4 User ID e06d572d4
  •  

janaton

Hi there,
I am main developer of the backend part of synchronization solution. First of all, I would like to thank everyone for participating in the first public test and his/her feedback (positive makes us happy, negative will be pushing us to improve as much as we can ;) ).

@tapio I fully understand your concern. All data are encrypted when transferred between client and server (https, REST API). We use Google Cloud Platform for all our services that are powering synchronization solution (storage encrypted internally by Google). We believe that security provided by Google is good enough. Additionally we try to do our best to have our code as much secure as possible (code review process, up to date libraries, vulnerability scans, ..). Still, no one can guarantee 100% security. Once you upload something to the internet it can be possibly hacked and exposed to third parties ... But again, our biggest concern is keeping your data safe - we are not selling it or sharing with third parties!

Client side encryption/decryption is not possible because we need to have access to it (e.g. web planner - access to your library from web browser, ability to plan track and sync to device, ..).

As already mentioned by Menion, selecting specific folders for sync is not possible (sync all or nothing). If you would like to participate in testing you could

  • Make backup, remove sensitive data and try sync, restore backup
  • Use Android emulator or another device with different DB. E.g. import points/tracks from public available GPX

Tapio

#7
Thx for explaining. Don't get me wrong, I trust you, my basic concern usually is, services get exploited and at some point we may find big data dumps available publicly - it always happens here and there... Well it's all about risk management,,, some stuff is OK for me to reside on the other end, other stuff only in encrypted form (with only me as the key owner).

But hey, since Alpha is having its own db here, I just remove "sensitive" folders and then sync.
Tapiola MFV4+ theme for OAM Maps:
Discuss - Releases - DL latest - Install latest
  •  
    The following users thanked this post: janaton

Menion

@balloni55
thanks for the log! Version 3.48.0.2/3 had a problem in methods that reported problem ::) so your log only contains info that the app crash when trying to report a problem  :).

New version MapGooglePlay_3.48.0.4_962_beta just uploaded.
- Official help (ideas, questions, problems): help.locusmap.eu
- Advanced topics, sharing of knowledges: you're here!
- LM 4 Beta download, LM 4 Release download
  •  

Tapio

OK I learned client side changes, also deletions, are synced to the server.

A crash happened.

- Everything was in sync.
- I deleted all tracks in a folder
- Locus started auto sync as set
- Locus disappeared
- Started it again, top left notification visible again, crash
- Then it was possible to start

Initial ideas.
a) Track/point/folder overlays if in sync
b) Different modes. "Upload only, download only, mirror device to cloud, mirror cloud to device"

Haha, thx for your effort, I guess you opened a can of worms here. 😉
Tapiola MFV4+ theme for OAM Maps:
Discuss - Releases - DL latest - Install latest
  •  

Tapio

#10
New version, it happened after track deletions.

And let me add: congrats for going public after probably a lot of hard work.
Tapiola MFV4+ theme for OAM Maps:
Discuss - Releases - DL latest - Install latest
  •  
    The following users thanked this post: Menion

Andrew Heard

#11
Quote from: menion on September 08, 2020, 13:06:05
Important: make a full backup of your database!! This version modifies the database upon start and also may cause a data loss in case of any serious problem in client-server communication.
@menion - is the database shared with LM3Pro? I don't want to install this version of LM4 if it modifies the LM3Pro database (unless it stays compatible). It's probably OK because balloni55 & tapio haven't expressed concerns.

Congrats too on this important milestone.
LM4.22.0 GOLD user ID:c7d47597a
  •  

Tapio

Not sure about the defaults. It's in Locus working directory (see misc settings).. /data/database/
I suggest you backup this folder. Or just create a backup.

My Locus instances use distinct paths.

@Menion, Locus UI does not display the  working dir. Just "SD card" is not enough.
Tapiola MFV4+ theme for OAM Maps:
Discuss - Releases - DL latest - Install latest
  •  

Menion

@tapio
may you please try you sync again? We identified and fixed issue on our server.

Ideas:
a) it needs a new "Data manager" screen (planned), but agree, will be useful
b) interesting idea, little dangerous. The question is if really needed. When the first sync after install happens, there is one alternative mode "clear device and download only", that should be enough I believe.

@Andrew Heard
as @tapio wrote, it depends on your settings (btw. full paths are under the "question mark" icon). In most cases, default settings for the Locus directory is currently the root of internal memory, so yes, databases are shared. It should not be a problem to use this shared database, but just to prevent any problems, suggest to keep backup in safe.
- Official help (ideas, questions, problems): help.locusmap.eu
- Advanced topics, sharing of knowledges: you're here!
- LM 4 Beta download, LM 4 Release download
  •  
    The following users thanked this post: Tapio

Tapio

I waited for an auto sync at first, but it did not do it. I think it should have tried it again.

Manual sync worked, thx.
Tapiola MFV4+ theme for OAM Maps:
Discuss - Releases - DL latest - Install latest
  •