Barnaby Puttick Posted April 29, 2019 Report Posted April 29, 2019 (edited) Hi, I seem to have discovered a bit of an issue using crossbar to retrieve ledger entries. There is a situation where crossbar does not return all expected records due to a bug when chunking is used (the default) Crossbar expects the "key" to be unique so it can pass the next_key to get the following chunked result and to determine EOF when last_key==next_key (chunking exhausted). cb_ledger use the "list_by_source" view to iterate through the records which has the "key" defined as [{service}, {pvt_created}] - however if there are 2 or more service entries that were created at the same time (multiple legs) this key is no longer unique and when this falls on a chunked boundary crossbar misinterprets this as EOF returning an incomplete result. This can be easily demonstrated by setting the chuck_size to 1 so that every entry is a chunk boundary and any duplicate key triggers a result exhaustion. For most part, this chunk exhaustion will not occur because the default chunk_size and page_size are both 50 resulting in a page exhaustion first. The question is, should this be fixed in crossbar's chunking algorithm or should the ledger view be changed to force a unique key? A temporary fix is to use is_chunked=false query parameter to disable chunking if you're experiencing missing records. Barnaby Edited April 29, 2019 by Barnaby Puttick (see edit history)
Barnaby Puttick Posted April 29, 2019 Author Report Posted April 29, 2019 This was tested on 4.3.29 and seems to be still prevalent in master and 4.3 (untested)
Recommended Posts