Syncing

[ErikBjare]

Vote on this issue on the forum!

---

There are two usage issues with ActivityWatch at the moment to which syncing is a solution:

• If you use more than one device, you need to check every device individually, or run one centralized instance of aw-server (not recommended!)
• If a machine is lost, so is the data (the user could have exported it, but data stored after the export would still be lost). While ActivityWatch cannot replace a proper backup system, syncing could help by storing copies of the data across devices.

I know of two interesting solutions to this problem:

• Centralized server which stores all data encrypted (the server is unable to decrypt)
  • Issues: Centralized, single point of failure
  • Done by @standardnotes
• P2P synchronization (encrypted, possibly including relays)
  • Done by @syncthing very well, perhaps we could use it in some way. Also: MPL2 licensed and written in Go.
    • Downside: Clients must be online at the same time for sync.
    • They have the ability to set some folders to "read only", useful when you want to ensure the data stays intact in its source.
  • Implementing it ourselves would be an enormous effort, I assume.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[ErikBjare]

@calmh might know a thing or two about using Syncthing in an application-specific context like this. I haven't seen it done before so we might want to check with him before we start.

I've taken a look at the arguments to Syncthing and found -home which can be used to set a custom configuration directory. So pretty promising.

[ErikBjare]

I've started prototyping something small here: https://github.com/ActivityWatch/aw-syncthing/

Could be made to work both with standalone Syncthing and bundled Syncthing, but standalone would probably be preferred due to the dependency on the Python package syncthing which targets a specific version (right now targets 0.14.24 and the latest is 0.14.25).

What it does:

• Moves the database file to a specific location
• Creates a symlink from the new to the old location (so aw-server will just follow the symlink to the file)
• Starts Syncthing with a custom configuration directory
• Configures Syncthing via the REST API to add the new database folder as a synced folder
• Add another device to sync the folder with

[calmh]

From a Syncthing point of view there's no real difference to it - you just start it up, point -home towards somewhere suitable, and configure it appropriately. You'll need to do the exchange of device IDs and so on somehow. As for the Python package, I see that it mentions 0.14.24 specifically but probably only because that was the latest when the README was written. All of 0.14.x speak the same API so there is no difference (and 0.12, 0.13 as well for the absolute most part).

[ErikBjare]

@calmh: Awesome! I'll let you know when we have a working release.

[ErikBjare]

I've started using Standard Notes recently (finally getting off Evernote) and have been impressed by the architecture. They have designed a neat data format/server called Standard File that defines how data should be encrypted and stored both client-side and server-side. Definitely something to check out.

Edit: It's interesting, but I'd rather have it distributed than just decentralized.

[ErikBjare]

I've been thinking about this a bit more.

My current idea is to simply configure a folder as a synced databases-folder. Basically aw-server would copy local data to this folder on a regular basis.

This folder could then be synced with Syncthing, Dropbox, or Gdrive (we should probably explicitly recommend Syncthing). The synced database files would not be allowed to be modified from another host then the one who owns them, since such changes could cause syncing conflicts.

Potential problems:

• It would be nice to have the synced databases encrypted
• Compressing them would also lead to huge storage savings

[ErikBjare]

Reddittracker turned up this today. Makes it pretty clear that sync is a vital feature for most users.

The best part is that you can put it on all computers (home and work) and on a smartphone. It'll track the software and sites you use on all of them and aggregate it to one account.

[hippylover]

Would be nice if this is implemented that it doesn't add to the system requirements to run the program. So for people who don't need the functionality or would rather just set up a cron job to copy it to a remote server manually can still disable the feature.

[ErikBjare]

@hippylover Noted! Thanks for the feedback.

[brizzbane]

I googled activitywatch + backup. Trying to locate where the data is stored. Would be really nice to be able to 'set' where the data is stored.

Backup solution I use--is to put important stuff I'm working on under Dropbox or MEGA. I'm under linux...and I actually add a home directory, so that I ...guess it makes me more 'aware' that it's Dropbox data.

I just read through the above comments--supposedly MEGA is end to end encrypted. I started using because more free data.. but it has the bonus of not having to mess with an encryption solution if you want to store it encrypted.