IteratorsIterators mk Tue, 04/28/2020 - 19:08
Many low level algorithms rely on an iterator to break a collection into smaller pieces. Each piece is subsequently processed by a block. For very large BATs it may make sense to break it into chunks and process them separately to solve a query. An iterator pair is provided to chop a BAT into fixed size elements. Each chunk is made available as a BATview. It provides read-only access to an underlying BAT. Adjusting the bounds is cheap, once the BATview descriptor has been constructed.
The smallest granularity is a single (oid,value) pair, which can be used to realize an iterator over the individual BAT elements. For larger sized chunks, the operators return a view.
All iterators require storage space to administer the location of the next element. The BAT iterator module uses a simple lng variable, which also acts as a cursor for barrier statements.
The larger chunks produced are currently static, i.e. their size is a parameter of the call. Dynamic chunk sizes are interesting for time-series query processing. (See another module)
module iterator; command new(b:bat[:oid,:any_2], size:lng) (:lng,:bat[:oid,:any_2]) address ITRnewChunk comment "Create an iterator with fixed granule size. The result is a view."; command next(b:bat[:oid,:any_2], size:lng) (:lng, :bat[:oid,:any_2]) address ITRnextChunk comment "Produce the next chunk for processing."; pattern new(b:bat[:oid,:any_2]) (h:oid, t:any_2) address ITRbunIterator comment "Process the buns one by one extracted from a void table."; pattern next(b:bat[:oid,:any_2]) (h:oid, t:any_2) address ITRbunNext comment "Produce the next bun for processing."; command next(step:oid,last:oid):oid address ITRnext_oid; command next(step:sht,last:sht):sht address ITRnext_sht; command next(step:int,last:int):int address ITRnext_int; command next(step:lng,last:lng):lng address ITRnext_lng; command next(step:flt,last:flt):flt address ITRnext_flt; command next(step:dbl,last:dbl):dbl address ITRnext_dbl comment "Advances the iterator with a fixed value";