[BUG] - gmvault_defaults.conf overwritten

[Stufo76]

My conf:

Python 2.7.3

Distributor ID: Debian
Description:    Debian GNU/Linux 7.10 (wheezy)
Release:        7.10
Codename:       wheezy

bash script in /root/bin:

gmail-backup.sh:
#!/bin/sh
/usr/local/bin/gmvault sync -type quick -d /root/my/db/folder myemail@mydomain.com

gmail-restore.sh
#!/bin/sh
/usr/local/bin/gmvault restore -type quick -d /root/my/db/folder myotheremail@mydomain.com

bash script in /etc/cron.hourly/ to run both above:
#!/bin/sh
test -x /root/bin/gmail-backup.sh || exit 0
/root/bin/gmail-backup.sh

#!/bin/sh
test -x /root/bin/gmail-restore.sh || exit 0
/root/bin/gmail-restore.sh

Hi all,

I have to backup and restore daily, so I tried to change /root/.gmvault/gmvault_defaults.conf

[Sync]
# quick_days=10
quick_days=1

[Restore]
# it is 10 days but currently it will always be the current month or the last 2 months
# the notion of days is not yet apparent in restore (only months).
# quick_days=10
quick_days=1
reserved_labels_map = { u'migrated' : u'gmv-migrated', u'\muted' : u'gmv-muted' }
...
...

but, if I run:

gmvault sync -type quick -d /my/db/folder myemail@mydomain.com

gmavault says:

Odd. Use default python2.7.x or 2.6.x distribution.

Use gmvault-db located in /root/my/db/folder

Activate Gmvault db cleaning.
Create defaults in /root/.gmvault/gmvault_defaults.conf. Please touch this file only if you know what to do.
Authentication performed with Gmail OAuth2 access token.

Get OAuth2 credential from /root/.gmvault/myemail@mydomain.com.

Connect to Gmail server.

Quick sync mode. Check for new emails since 17-Apr-2016.
Start synchronization (2016-04-27T11h41m56s).

Start emails synchronization.
355 emails to be fetched.
Process email num 0 (imap_id:15043) from 2016-04.
Process email num 1 (imap_id:15044) from 2016-04.
Process email num 2 (imap_id:15045) from 2016-04.
Process email num 3 (imap_id:15046) from 2016-04.
Process email num 4 (imap_id:15047) from 2016-04.
Process email num 5 (imap_id:15048) from 2016-04.
Process email num 6 (imap_id:15049) from 2016-04.
...
...
...
================================================================
Sync operation performed in 4s.
Number of reconnections: 0.
Number of emails quarantined: 0.
Number of emails that could not be fetched: 0.
Number of emails that were returned empty by gmail: 0.
Number of emails without label information returned by gmail: 0.
================================================================

and $HOME/.gmvault/gmvault_defaults.conf was overwritten with defaults.

Same for restore.

How can I permanent change sync and restore of last 24 hours?

Or, can I pass arguments to gmvault in order to have same result?

Thank you very much.

[LeoFCardoso]

This is a bad behavior of this great app.
Workaround is playing with "custom" type of sync and use "imap_req". For instance (as in docs):

$> gmvault sync --type custom --imap-req 'Since 1-Nov-2011 Before 10-Nov-2011' 'foo.bar@gmail.com' -c no

[lalinsky]

This is fixed here - #273

[gaubert]

@lalinsky @LeoFCardoso @Stufo76 you can test the latest beta version here (beta windows and mac os x package available):
https://www.dropbox.com/sh/d5ceo77juacr03y/AACUGcTt6Um-6j6JmBizGPA2a?dl=0

or from the branch gmv-1.9.2

Help required for testing to see if I missed something. Many thanks for the testing.

[gaubert]

@Stufo76 @lalinsky @LeoFCardoso version 1.9.2-beta-1 available for test and fixing many additional problems (especially in the restore). Please test it and let me know if this is fine within 6 days. Many thanks.

https://www.dropbox.com/sh/d5ceo77juacr03y/AACUGcTt6Um-6j6JmBizGPA2a?dl=0

[aberja]

@gaubert , I've been using 1.92-beta-1 and it appears to have fixed the gmvault_defaults.conf overwrite bug.

However, a couple of times since installing this beta version I have experienced the following error when doing a full sync using gmvault sync -d FolderName EmailName@gmail.com -c no (and then after the 1st crash: gmvault sync --resume -d FolderName EmailName@gmail.com -c no ):

Error: [Errno -1] Unexpected EOF. The crashes occurred after email num 31935 & 126066

I have never experienced this error before with 1.91. Here is a screenshot of the Exception traceback:

https://www.dropbox.com/s/e3ifr2hf14imo0c/2017-02-14%2009_14_34-GMVAULT-SHELL.png?dl=0

[gaubert]

@aberja ok thanks a lot I have encountered that problem during my testing. I need to check.

[gaubert]

@aberja there seem to be problems with the python ssl support in the imap lib.
I could try to catch the exception for a smooth restart at least. It is some kind of hipcup in the ssl exchange. What do you think ?

[aberja]

@gaubert Yes, I think that makes sense. It would be nice if gmvault would actually --resume automatically after the error. I've done more testing over the last couple of days and I only see the error when syncing emails >10k. This could be a coincidence, but I thought I would mention it just in case it helps.

[aberja]

@gaubert As I was testing the beta version I came across another error:

Error: Problems when trying to connect to Google oauth2 endpoint: https://accounts.google.com/o/oauth2/token
Error: HTTP Error 400: Bad Request.

The Screenshot of the error can be found here: https://www.dropbox.com/s/yixx2zo4srv9wl5/2017-02-18%20GMVAULT-SHELL%20Error%20400.png?dl=0

The error occurs when one adds 2 step authorization to a Gmail account after having set the account up in gmvault. I'm not sure why this occurs because I thought using oauth got around this type of issue. Perhaps Google terminates all existing tokens when it adds 2 step authorization?

I had a Gmail account without 2 step authorization which I was syncing with gmvault for the past couple of months. Today I decided to add 2 step authorization and see what would happen with gmvault. The result was an error 400 and no ability to continue syncing the account.

[aberja]

Just a quick follow-up to my last post about the oauth2 error. I worked around the issue by deleting the respective .oauth2 file in gmvault/.gmvault and rerunning the sync command in gmvault. gmvault then obtained a new token and I was able to sync the account.