S3 Storage plan MODB & Account

[abuzooz]

Hi All

I am using S3 storage plan to save recordings, media and voice messages, however, I noticed that kazoo is only saving a copy of the file binary to S3 and keeping another copy in Couchdb, the local copy is used by Kazoo when once of these files is requested based on the logs.

I was reading and I noticed that connection should be configured not to use local (couchdb), howver,  there is no documentation for that in the web that I can find to change the connection to s3, any advise? is there any drawbacks/concerns not saving a local copy of these binaries? like performance..etc?

 

{
   "data":{
      "attachments":{
         "bb966f5b0af243bb835b6c07ca3120a7":{
            "handler":"s3",
            "name":"Kazoo S3",
            "settings":{
               "bucket":"XX",
               "key":"XXX",
               "secret":"XXXXX/XXXXX",
               "bucket_access_method":"auto",
               "field_list":[
                  {
                     "arg":"account_id"
                  },
                  {
                     "const":"voicemail"
                  },
                  {
                     "arg":"attachment"
                  }
               ]
            }
         },
         "2e878cb4a14a46d4abd35a9a3ce6ae8e":{
            "handler":"s3",
            "name":"Kazoo S3",
            "settings":{
               "bucket":"XX",
               "key":"XXX",
               "secret":"XXXX/XXXXX",
               "bucket_access_method":"auto",
               "field_list":[
                  {
                     "arg":"account_id"
                  },
                  {
                     "const":"media"
                  },
                  {
                     "arg":"attachment"
                  }
               ]
            }
         },
         "f8885bb42bdb4466a88847e2d6fbe1b2":{
            "handler":"s3",
            "name":"Kazoo S3",
            "settings":{
               "bucket":"XX",
               "key":"XXX",
               "secret":"XXX/XXXX",
               "bucket_access_method":"auto",
               "field_list":[
                  {
                     "arg":"account_id"
                  },
                  {
                     "const":"recordings"
                  },
                  {
                     "arg":"attachment"
                  }
               ]
            }
         }
      },
      "plan":{
         "modb":{
            "attachments":{
               
            },
            "connection":"local",
            "types":{
               "call_recording":{
                  "attachments":{
                     "handler":"f8885bb42bdb4466a88847e2d6fbe1b2"
                  }
               },
               "mailbox_message":{
                  "attachments":{
                     "handler":"bb966f5b0af243bb835b6c07ca3120a7"
                  }
               },
               "media":{
                  "attachments":{
                     "handler":"2e878cb4a14a46d4abd35a9a3ce6ae8e"
                  }
               }
            }
         },
         "account":{
            "attachments":{
               
            },
            "connection":"local",
            "types":{
               "call_recording":{
                  "attachments":{
                     "handler":"f8885bb42bdb4466a88847e2d6fbe1b2"
                  }
               },
               "mailbox_message":{
                  "attachments":{
                     "handler":"bb966f5b0af243bb835b6c07ca3120a7"
                  }
               },
               "media":{
                  "attachments":{
                     "handler":"2e878cb4a14a46d4abd35a9a3ce6ae8e"
                  }
               }
            }
         }
      }
   }
}

Edited October 28, 2020 by abuzooz (see edit history)

[mc_]

How are you configuring call recording to occur? By configs on account/user/device or explicitly in the callflow ?

[abuzooz]

@mc_

on user level and  by using below API json payload. Call recording is sent successfully to S3 bucket with no issues, however, I feel that MODB is still saving a copy of that file.

 

{  "auth_token": "{AUTH_TOKEN}",

"data":{

    "smartpbx":{"call_recording":{"enabled":true}},

"call_recording":{

    "outbound":{

        "offnet":{"record_on_answer":true,"record_min_sec":1,"enabled":true,"time_limit":3600,"format":"mp3"},

        "onnet":{"enabled":false}},

    "inbound":{

        "offnet":{"record_on_answer":true,"record_min_sec":1,"enabled":true,"time_limit":3600,"format":"mp3"},

        "onnet":{"enabled":false}}}}

     }

     

     

[mc_]

Well, you would need to check the document in couch to be sure. It should contain all the metadata about the recording but should not have an '_attachments' key with the binary data.

[abuzooz]

@mc_ I will check, but why I am sure it is saving also a copy of the metadata in couchdb  as for example in voice message case, I deleted 1 of voice messages from S3 and kazoo was able to play it normally even if the file doesnt exist in S3, the stream came from couchdb.

 

[mc_]

FreeSWITCH caches the recording so it could be your cache didn't time out yet, but please do check. We don't want to be redundant in saving the binary!

[abuzooz]

here is couchdb document for your reference, I can see that it says s3, so this means it is only stored on S3?

 

{
   "_id": "202010-d471852c51ffad66d329649c6895684a",
   "_rev": "2-c66055d1bcdba32d469141db5c353afd",
   "pvt_document_hash": "558c42991392dc1d091f694152de7ad0",
   "pvt_node": "kazoo_apps@kz.xxxx.com",
   "pvt_type": "call_recording",
   "pvt_modified": 63771302492,
   "pvt_created": 63771302492,
   "pvt_account_db": "account%2Fba%2F47%2F74842e5840b29c5f6ab1784ad171-202010",
   "pvt_account_id": "ba4774842e5840b29c5f6ab1784ad171",
   "to": "972599796989@atlantic.xxxx.com",
   "start": 63771302488,
   "source_type": "kzc_recording",
   "request": "972599796989@xxxx.xxxx.com",
   "owner_id": "2399970e0db1a68698fd21ea0ec1999c",
   "origin": "outbound to offnet from endpoint",
   "name": "69ba68f484264a005a22eb49838bfce0.mp3",
   "media_type": "mp3",
   "media_source": "recorded",
   "interaction_id": "63771302480-fd575c22",
   "from": "1008@atlantic.deskwaves.com",
   "duration_ms": 3520,
   "duration": 3,
   "direction": "inbound",
   "description": "recording 69ba68f484264a005a22eb49838bfce0.mp3",
   "custom_channel_vars": {
       "Account-ID": "ba4774842e5840b29c5f6ab1784ad171",
       "Account-Name": "xxxx",
       "Account-Realm": "atlantic.xxxx.com",
       "Application-Name": "callflow",
       "Application-Node": "kazoo_apps@kz.xxxxx.com",
       "Authorizing-ID": "ed4421e3c7d6d66f303279edce5cd322",
       "Authorizing-Type": "device",
       "Bridge-ID": "169d74e3843f406bbd52a1e5490d54f1",
       "Call-Interaction-ID": "63771302480-fd575c22",
       "CallFlow-ID": "25178a8c5f2ad05d145e59f89835679d",
       "Channel-Authorized": "true",
       "Ecallmgr-Node": "ecallmgr@ecallmgr.xxxxx.com",
       "Fetch-ID": "5a3cff51-9776-4455-ab8f-3da35ef6bb58",
       "Media-Name": "69ba68f484264a005a22eb49838bfce0.mp3",
       "Media-Names": "69ba68f484264a005a22eb49838bfce0.mp3",
       "Media-Recorder": "kz_media_recording",
       "Media-Recording-ID": "202010-d471852c51ffad66d329649c6895684a",
       "Media-Recordings": "202010-d471852c51ffad66d329649c6895684a",
       "Owner-ID": "2399970e0db1a68698fd21ea0ec1999c",
       "Privacy-Hide-Name": "false",
       "Privacy-Hide-Number": "false",
       "Realm": "xxxxx.xxxxx.com",
       "Reseller-ID": "d751fcae7bebadcae4acfa188ca21939",
       "Username": "yazan"
   },
   "content_type": "audio/mpeg",
   "cdr_id": "202010-169d74e3843f406bbd52a1e5490d54f1",
   "caller_id_number": "1008",
   "caller_id_name": "Yazan Inaimss",
   "callee_id_number": "972599796989",
   "callee_id_name": "",
   "call_id": "169d74e3843f406bbd52a1e5490d54f1",
   "pvt_attachments": {
       "69ba68f484264a005a22eb49838bfce0.mp3": {
           "content_type": "audio/mpeg",
           "length": 30758,
           "stub": false,
           "handler": {
               "kz_att_s3": {
                   "S3": "g2gCdAAAAAVkAAZidWNrZXRtAAAACXRhbGtjaGllZmQAFGJ1Y2tldF9hY2Nlc3NfbWV0aG9kbQAAAARhdXRvZAAKZmllbGRfbGlzdGwAAAAEdAAAAAFtAAAAA2FyZ20AAAAKYWNjb3VudF9pZHQAAAABbQAAAAVjb25zdG0AAAAKcmVjb3JkaW5nc3QAAAABbQAAAANhcmdtAAAAAmlkdAAAAAFtAAAAA2FyZ20AAAAKYXR0YWNobWVudGpkAANrZXltAAAAFEFLSUE0VDVUSktFQUlXNVZVVUlNZAAGc2VjcmV0bQAAAChOMml3WUhmVHZFOEFISXVXcTQvM3haWUpZRmVBcVdXS3Jpc1JKOEtKbQAAAHhiYTQ3NzQ4NDJlNTg0MGIyOWM1ZjZhYjE3ODRhZDE3MS9yZWNvcmRpbmdzLzIwMjAxMC1kNDcxODUyYzUxZmZhZDY2ZDMyOTY0OWM2ODk1Njg0YS82OWJhNjhmNDg0MjY0YTAwNWEyMmViNDk4MzhiZmNlMC5tcDM=",
                   "metadata": {
                       "etag": "170b49ee20700105af1e0ec2a850ce42",
                       "x-amz-request-id": "FDF91953B94C0266",
                       "x-amz-id-2": "FgwO599YIaz4c5D96A89hLaS4/pRg68jqQYA7OpFISk0smZ+mz6kxDPXaZHJ1HAb0Jb2zcAOYUc="
                   }
               }
           }
       }
   }
}

[mc_]

Correct, "pvt_attachments" is a KAZOO-added thing for the metadata. "_attachments" is the Couch key for binary data attached to the document. No "_attachments", no recording data

[abuzooz]

All is good then, thanks for the clarification!