Jump to content
KAZOOcon: hackathon signup and details here! ×

Bugs in couchdb-dump-php!


Recommended Posts

Hi!
I don't know if this is the right forum, but I'll try anyway.

A while ago I noticed some bugs in the couchdb-dump-php tools (https://github.com/2600hz/couchdb-dump-php).
I didn't see the impact of this until today, when accidentally removed my company's whole database in Couch(!) (oops, took 3 seconds with a faulty curl command, and 6-7 hours to restore from backup).

We're using couchdb-dump.php to make nightly backups of the whole CouchDB (https://forums.2600hz.com/forums/topic/9275-couchdb-backup-strategies/?do=findComment&comment=51413)
However, it seems like the restore script imports the objects as strings instead!
In devices documents, ' "caller_id": { }, '' gets imported as ' "caller_id": [ ], ', same with contact_list etc and also in users documents. To solve this you can use the sed commands below, in the backup file before restoring it.

sed -i -e 's/caller_id\":\[\]/caller_id\":\{\}/g' /tmp/backup.json
sed -i -e 's/contact_list\":\[\]/contact_list\":\{\}/g' /tmp/backup.json
sed -i -e 's/dial_plan\":\[\]/dial_plan\":\{\}/g' /tmp/backup.json
sed -i -e 's/contact_list\":\[\]/contact_list\":\{\}/g' /tmp/backup.json
sed -i -e 's/music_on_hold\":\[\]/music_on_hold\":\{\}/g' /tmp/backup.json
sed -i -e 's/ringtones\":\[\]/ringtones\":\{\}/g' /tmp/backup.json
sed -i -e 's/call_restriction\":\[\]/call_restriction\":\{\}/g' /tmp/backup.json
sed -i -e 's/profile\":\[\]/profile\":\{\}/g' /tmp/backup.json
sed -i -e 's/media\":\[\]/media\":\{\}/g' /tmp/backup.json


Another thing that doesn't work as expected are the phone numbers. The plus sign is imported into space instead;
image.png.f9d6c3e7116e8da755dbb0cfdf6704a6.png

To solve this you need to copy, then delete and create a new number doc and replace space with plus, then save. On every number, manually...


We tried to look into the code of the scripts, but can't see why this bugs occur. Hopefully some more experienced developers out there can fix this!

Edited by tomas_ (see edit history)
Link to comment
Share on other sites

  • 9 months later...

Hi!
Today it happened again, this time with bigger impact - Many databases discarded when an old, removed couchdb node (with old databases) went online after a power outage and somehow synced the old database with the other nodes. (Not good at all, you learn something each day! ;) ).
(Since time we've started to make tar backups of the /srv folders on each db node also, but the last few days the disks went full and our automatic VPS script to increase disk storage wasn't working. Bad luck!!)

So we started to restore databases which was backed up by our script running couchdb-dump, but then we ran into the issues in my post above!
The number docs was easily fixed, just replace the + sign in the number/doc id with %2B. Sed is a really good tool, to fix we just ran:

sed -i -e 's/_id":"+/_id":"%2B/g' number*

 

However, the issues still remains. Anyone that can see why?
There must be other out there using the couchdb-dump tool which has the same issues?

Br - Tomas

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...