EPIM Synchronizer and Archiver commandline

Your suggestions for future versions of EssentialPIM

Moderators: TerryRogers, Max

Triple S
Expert
Posts: 70
Joined: Wed Oct 21, 2020 5:07 pm
Been thanked: 7 times

EPIM Synchronizer and Archiver commandline

Post by Triple S »

I would like to be able to use both the EPIM Synchronizer and EPIM Archiver tool with the command line so that I can make automated tasks out of them.

Alternative suggestion:
Command line options for EPIM to open a database, and run an export job.

Alternative suggestion #2:
A way to configure backups per module.
Each module should be able to have it's own frequency, folder, and maximum number of backups.

Reasons:
I only want backups of my notes. I don't care about backups for any of my other data, especially email as they simply make backups needlessly heavy, and are already backed up on the email server.

As of now I have to make note backups manually, which is time consuming, and prone to human failure.
admin
Site Admin
Posts: 15601
Joined: Thu Nov 25, 2004 3:12 am
Has thanked: 1411 times
Been thanked: 984 times

Re: EPIM Synchronizer and Archiver commandline

Post by admin »

Thanks for the suggestions, noted. I think the easiest and the most useful one for the majority of users would be "Alternative suggestion #2".
Android version of EssentialPIM. Keep all your data in sync!
SilverSound
Expert
Posts: 73
Joined: Tue Jun 09, 2020 6:46 am
Been thanked: 4 times

Re: EPIM Synchronizer and Archiver commandline

Post by SilverSound »

Alternative suggestion #2, certainly sounds like the easiest, though, I can see advantages to the original suggestion, especially setting up a windows task for EPIM Archiver.

It would be especially handy to be able to auto archive email, task, and calendar content automatically, perhaps that could be a function built into EPIM to make it more accessible though, as the windows task manager is a scary place for most users in my experience.
admin
Site Admin
Posts: 15601
Joined: Thu Nov 25, 2004 3:12 am
Has thanked: 1411 times
Been thanked: 984 times

Re: EPIM Synchronizer and Archiver commandline

Post by admin »

Actually this is already possible via using the sync with EPIM server feature. The only thing you need to make it work is locally installed Firebird server (you can download the Business edition and install the server with it). Then set up sync into a new (backup) database, choose the kind of data you'd like to transfer, set up sync (backup) interval and you should be good to go.
Android version of EssentialPIM. Keep all your data in sync!
SilverSound
Expert
Posts: 73
Joined: Tue Jun 09, 2020 6:46 am
Been thanked: 4 times

Re: EPIM Synchronizer and Archiver commandline

Post by SilverSound »

admin wrote: Sat Dec 11, 2021 10:24 am Actually this is already possible via using the sync with EPIM server feature. The only thing you need to make it work is locally installed Firebird server (you can download the Business edition and install the server with it). Then set up sync into a new (backup) database, choose the kind of data you'd like to transfer, set up sync (backup) interval and you should be good to go.
I have used the sync to EPIM server for quite a while and it has two major flaws compared to the archiver.

#1 It does not delete data that it synced. The main point of the archiver IMO is to remove old data, which the inbuilt sync is missing entirely.

#2 It is clunky and unoptimized. The EPIM syncronizer and Archiver tools are lighting fast even over a network with huge databases and virtually never cause issues. Where as the sync to EPIM sever is extremely slow, even on the same computer, prone to strange data errors and corruption, and is too ridged in it's scheduling; using the windows task manager you can pick your days, have the system be idle...etc EPIM sync to server requires EPIM to be open, and only offers a an option to sync every x minutes with no idle wait, no specific days...etc. EPIM sever sync is overall incredibly limited compared to the windows task scheduler.
admin
Site Admin
Posts: 15601
Joined: Thu Nov 25, 2004 3:12 am
Has thanked: 1411 times
Been thanked: 984 times

Re: EPIM Synchronizer and Archiver commandline

Post by admin »

1. EPIM Server Sync feature purpose is to sync data, not to remove it. If you wish to be able to remove your old data automatically, mention the type of data you'd like to remove (there's built into EPIM archive feature for mail, for instance), how often and why. What's your use case scenario?
2. Comparing EPIM Servers sync feature with EPIM Archiver/DupeRemover is like comparing apples and oranges. We'd also like to see some proof about errors and corruptions and sync slowness.
Android version of EssentialPIM. Keep all your data in sync!
SilverSound
Expert
Posts: 73
Joined: Tue Jun 09, 2020 6:46 am
Been thanked: 4 times

Re: EPIM Synchronizer and Archiver commandline

Post by SilverSound »

admin wrote: Sun Dec 12, 2021 2:18 pm 1. EPIM Server Sync feature purpose is to sync data, not to remove it. If you wish to be able to remove your old data automatically, mention the type of data you'd like to remove (there's built into EPIM archive feature for mail, for instance), how often and why. What's your use case scenario?
2. Comparing EPIM Servers sync feature with EPIM Archiver/DupeRemover is like comparing apples and oranges. We'd also like to see some proof about errors and corruptions and sync slowness.
1.a I want to be able to archive Mail, Calendar, and tasks that are 3 years old ever 3 months.
1.b I would also like to do a sync to my laptop every week on Sunday at 10:00pm.

2. I fully agree, you were the one that was trying to tell us they were comparable and I was telling you that it was apples and oranges so I'm glad you see the issues with your own comparison.

You will have to find someone else to provide proof, I do not have the time, but I can assure you there are many issues that crop up, most are one off flukes and would require immense amounts of time to provide details on, but since the one off flukes are not rare enough to be wholly brushed off they pile up over long term use.

Flukes off the top of my head:

Data loss. Two-way-syncs are very prone to strange one off data loss. I've had to restore data from local databases many times because the sync decided it wanted to cause some kind of damage. The data loss has been very wide, deleted contacts, notes that don't save, tasks that don't have the details overwritten with old versions...etc.

Icons, sever sync for some reason likes to change note icons sometimes.

General slowness, sever sync can be simply slow and hang for many minutes even if nothing has changed. Even when both sever and client are on high end hardware or the same system.
admin
Site Admin
Posts: 15601
Joined: Thu Nov 25, 2004 3:12 am
Has thanked: 1411 times
Been thanked: 984 times

Re: EPIM Synchronizer and Archiver commandline

Post by admin »

1.a. Ok, let's see if there's more requests like this one.
1.b. Why not use EPIM Cloud for that?

2. There's no issues, that's the way users can already use to transfer their pre-selected data types into another database at pre-defined period of times.
Android version of EssentialPIM. Keep all your data in sync!
SilverSound
Expert
Posts: 73
Joined: Tue Jun 09, 2020 6:46 am
Been thanked: 4 times

Re: EPIM Synchronizer and Archiver commandline

Post by SilverSound »

admin wrote: Mon Dec 13, 2021 9:48 am 1.a. Ok, let's see if there's more requests like this one.
1.b. Why not use EPIM Cloud for that?

2. There's no issues, that's the way users can already use to transfer their pre-selected data types into another database at pre-defined period of times.

1.b My Laptop is an offline only device, that's the whole reason I use EPIM.

2. You don't seem to be interested in the faults of EPIM server sync, so let's just leave it at it simply does not fit my needs nor desires, whereas the EPIM archiver fits all of them perfectly, but is not lacks scheduling because of like of cmd interface.
MetalDrop
Guru
Posts: 866
Joined: Sat Apr 09, 2016 10:19 pm
Been thanked: 189 times

Re: EPIM Synchronizer and Archiver commandline

Post by MetalDrop »

+1 Would love to have command line options for Synchronizer and Archiver. I'd be able to make bat files for a lot of maintenance tasks, and tying them into windows scheduler would certainly be handy too.

Side note: I use EPIM sever sync frequently, and it does have a lot of data loss issues at least in 2-way-sync, but they are hard to track down and I haven't been able to report any of them due to lack of reproducible steps.

One major issue that has never been fixed is proper conflict handling which also makes it hard, since if two people are syncing to the same server that can cause all sort strange data loss, but again very hard to reproduce, I can't make any solid steps right now. The only thing I can say is:

Set up EPIM sever sync and have two people sync data both ways to/from it and use it a lot and data loss will happen randomly, especially notes... not very helpful, but that's also why I haven't reported any of it yet.
EPIM Portable Pro Running/Tested On: Windows 11 Pro 64-bit US-ENG|i5-6400+Quadro P620|i7-7700K+1050ti|i7-8700K+970GTX|AMD 5600x+1080ti|16GB+RAM&NVMe SSDs
[I'm helpful and often reply to questions, however I am just a fellow user and not staff.]
admin
Site Admin
Posts: 15601
Joined: Thu Nov 25, 2004 3:12 am
Has thanked: 1411 times
Been thanked: 984 times

Re: EPIM Synchronizer and Archiver commandline

Post by admin »

All right, to sum up. I suppose you expect the following command line parameters to be supported by EPIM Sync:
- source/target database file + password for local databases and in addition to that network login + network password for the DBs in the LAN
- sync direction
- data types (modules)

For EPIM Archiver:
- source/target database file + password for local databases and in addition to that network login + network password for the DBs in the LAN
- data starting with which the older items are to be deleted
- action (archive to another DB, delete to trash, delete permanently)

In addition to that we can make a dialog window that would allow you to easily set up scheduled syncs (to be executed with Win Scheduler) directly from the apps (no need to use command line).
The question is - how many people need this. Let's wait and find out.

About EPIM Server sync - thanks for the info, will investigate this.
Android version of EssentialPIM. Keep all your data in sync!
FireBrand2026
Guru
Posts: 142
Joined: Fri May 13, 2016 4:10 am
Been thanked: 9 times

Re: EPIM Synchronizer and Archiver commandline

Post by FireBrand2026 »

admin wrote: Mon Dec 13, 2021 9:42 pm In addition to that we can make a dialog window that would allow you to easily set up scheduled syncs (to be executed with Win Scheduler) directly from the apps (no need to use command line).
The question is - how many people need this. Let's wait and find out.
I'm not comfortable with setting things up in the windows scheduler, but would get some good use from running the archiver automatically outside of EPIM. So if I'm reading this right and a dialog could be made to setup the windows scheduler to run it, that would be amazing and a big +1 from me.
admin
Site Admin
Posts: 15601
Joined: Thu Nov 25, 2004 3:12 am
Has thanked: 1411 times
Been thanked: 984 times

Re: EPIM Synchronizer and Archiver commandline

Post by admin »

Yep, that's what I meant. Ok, your "+1" counted towards the feature :)
Set up EPIM sever sync and have two people sync data both ways to/from it and use it a lot and data loss will happen randomly, especially notes... not very helpful, but that's also why I haven't reported any of it yet.
By data loss you mean that only the latest changes are available after the sync? That's how it supposed to work. And with the 10.0 version you will be able to see who made the latest changes and when and to revert back to the previous version, if needed.
Android version of EssentialPIM. Keep all your data in sync!
MetalDrop
Guru
Posts: 866
Joined: Sat Apr 09, 2016 10:19 pm
Been thanked: 189 times

Re: EPIM Synchronizer and Archiver commandline

Post by MetalDrop »

Set up EPIM sever sync and have two people sync data both ways to/from it and use it a lot and data loss will happen randomly, especially notes... not very helpful, but that's also why I haven't reported any of it yet.
By data loss you mean that only the latest changes are available after the sync? That's how it supposed to work. And with the 10.0 version you will be able to see who made the latest changes and when and to revert back to the previous version, if needed.
[/quote]

No, often an old version will replace the new version with no warning. It seems to happen more often when people read a note that someone else has edited, but sometimes it doesn't even need the other person to read the note.

I can't get it to repeat consistently but what can happen the most often seems to be:

Person A: Writes a note, saves and syncs the note, leaves their computer on and goes away from the computer.

Person B: Syncs to get Persons A note, they read the note, then they go back to work.

Person A: Comes back starts writing in the note some more then syncs it at the end of the day.

Person B: Who never changed Persons A note syncs their data at the end of the day after Person A, and persons A note is reverted to the state of their sync before lunch.

Person A: Starts the day by syncing only to find that everything they have done since lunch the day before has been lost, and has to restore it from a local backup.

I've had other strange things happen, but that's the closest one I've gotten to pinning down.
I obviously haven't been using 10 seriously yet only in light testing looking for obvious bugs and issues.
EPIM Portable Pro Running/Tested On: Windows 11 Pro 64-bit US-ENG|i5-6400+Quadro P620|i7-7700K+1050ti|i7-8700K+970GTX|AMD 5600x+1080ti|16GB+RAM&NVMe SSDs
[I'm helpful and often reply to questions, however I am just a fellow user and not staff.]
admin
Site Admin
Posts: 15601
Joined: Thu Nov 25, 2004 3:12 am
Has thanked: 1411 times
Been thanked: 984 times

Re: EPIM Synchronizer and Archiver commandline

Post by admin »

Thanks very much for such a detailed lists of steps, it will help us a lot with debugging the issues.
Android version of EssentialPIM. Keep all your data in sync!
Post Reply