ToC path:

Sequence Iterator

The public C sequence interface uses mco_seq_iterator_t, declared in mcoseq.h, both as a cursor over stored sequence values and as a node in a lazy sequence-expression pipeline.

    typedef struct mco_seq_iterator_t_  mco_seq_iterator_t;
    typedef mco_seq_iterator_t*         mco_seq_iterator_h;

The complete structure is public so that applications can allocate iterators without an additional heap allocation and advanced code can process tiles directly. Most fields contain implementation state. Applications should initialize an iterator to zero, pass its address to an iterator constructor, and use public sequence functions to consume or reset it.

Iterator Construction and Lifetime

An iterator constructor can be a generated sequence-field function such as classname_fieldname_iterator(), or a generic operation such as mco_seq_add_float(). Generic operations normally store pointers to their operand iterators rather than copying or materializing them.

Keep every operand iterator alive and unchanged until the result iterator has been completely consumed.
Keep the database transaction and object handle used by a generated field iterator alive while that iterator is in use.
Keep callback contexts and application buffers referenced by an iterator alive for the same period.
Use a separate source iterator for each independent consumer, or use mco_seq_tee().
Do not copy an initialized iterator with memcpy.

Typed Logical Traversal

The preferred traversal interface is mco_seq_next_TYPE():

    MCO_RET mco_seq_next_TYPE(mco_seq_iterator_h iterator,
            mco_TYPE* value);

TYPE is one of the scalar type suffixes supported by the operation. A successful call returns MCO_S_OK and one logical sequence element. MCO_S_CURSOR_END indicates normal exhaustion. For a NULL element, the function returns MCO_E_SEQ_NULL_VALUE, advances the iterator, and leaves the payload unspecified.

Typed traversal has the same logical behavior with plain and RLE sequence libraries. The RLE implementation expands each run so that the application receives one value per logical element.

Reset an Iterator Pipeline

Iterator constructors install the correct reset callback for the iterator and its operand shape. Use that callback when a pipeline must be traversed again:

    MCO_RET rc;

    if (iterator.reset == NULL)
        rc = MCO_E_ILLEGAL_PARAM;
    else
        rc = iterator.reset(&iterator);

The public functions mco_seq_reset_iterator(), mco_seq_reset_unary_iterator(), and mco_seq_reset_binary_iterator() implement the standard reset callbacks for source, unary, and binary iterators. They are useful when implementing a custom iterator. Normal application code should call the callback already installed in the iterator instead of selecting one of these functions itself.

Direct Tile Traversal

Direct tile traversal avoids one function call per logical element but couples the application to the physical representation. Call the iterator's next callback to load a tile. The following fields are relevant after next returns MCO_S_OK:

elem_type and elem_size

Describe the physical element type and width.

tile_size

Number of physical items in the current tile. For RLE, this is the number of runs, not the number of logical values.

tile.u.arr_TYPE

Typed payload array for the current tile.

tile.null_bitmap

NULL bits for the physical tile items. A set bit denotes NULL.

first_seq_no, next_seq_no, and last_seq_no

Logical sequence boundaries and traversal position. Treat them as read-only state.

The current tile container is declared as follows:

    typedef union mco_seq_payload_t_ {
        /* arr_int1, arr_int2, ..., arr_datetime,
           arr_float, arr_double, and arr_char */
    } mco_seq_payload_t;

    typedef struct mco_seq_tile_t_ {
        mco_bitmap_word_t null_bitmap[MCO_SEQ_NULL_BITMAP_SIZE];
        mco_seq_payload_t u;
    } mco_seq_tile_t;

The following example correctly sums a float iterator with either sequence implementation. With plain sequences, mco_seq_rle_count() returns 1. With RLE, it returns the logical multiplicity of the physical run:

    mco_uint2 i;
    mco_seq_rle_count_t count;
    double sum = 0.0;
    MCO_RET rc;

    while ((rc = iterator.next(&iterator)) == MCO_S_OK)
    {
        for (i = 0; i < iterator.tile_size; ++i)
        {
            count = mco_seq_rle_count(&iterator, (int)i);
            if (!MCO_SEQ_IS_NULL(&iterator, i))
                sum += (double)iterator.tile.u.arr_float[i] * count;
        }
    }
    if (rc == MCO_S_CURSOR_END)
        rc = MCO_S_OK;

Do not zip independently compressed RLE fields by physical tile index. Their run boundaries can differ even when their logical lengths are equal. Use typed logical traversal, decode each stream, or use an RLE-aware multi-input sequence operator.

Iterator Callback Types

The callback fields use the following public types:

    typedef MCO_RET (*mco_seq_iterator_next_t)(
            mco_seq_iterator_h iterator);
    typedef MCO_RET (*mco_seq_iterator_reset_t)(
            mco_seq_iterator_h iterator);
    typedef MCO_RET (*mco_seq_iterator_prepare_t)(
            mco_seq_iterator_h iterator);
    typedef MCO_RET (*mco_seq_iterator_merge_t)(
            mco_seq_iterator_h dst,
            mco_seq_iterator_h src);

next produces the next physical tile, reset rewinds local and operand state, and prepare and merge support operators that can be evaluated in parallel. Applications consuming standard iterators call the installed callbacks; only custom iterator implementations assign them.

Fields Reserved for Iterator Implementations

The callback pointers next, reset, prepare, and merge; operand pointers; context storage; buffer ownership flags; RLE offsets; parallel-processing fields; and binding state are maintained by sequence constructors. Do not modify them unless implementing a custom iterator according to the contracts in mcoseq.h.

Some operators allocate temporary memory, and some hash operators maintain separate hash state. Follow the lifecycle documented for the constructing function. mco_seq_free_buffer() releases only a buffer owned by the specified iterator and does not recursively release its operands.

Sequence Boundaries

Generated search and range functions use mco_seq_boundary_kind_t:

    typedef enum mco_seq_boundary_kind_t_ {
        MCO_SEQ_BOUNDARY_OPEN,
        MCO_SEQ_BOUNDARY_INCLUSIVE,
        MCO_SEQ_BOUNDARY_EXCLUSIVE
    } mco_seq_boundary_kind_t;

An open boundary omits that end of the range. Inclusive and exclusive boundaries include or exclude the specified key. Sequence positions and boundaries are logical element positions for both plain and RLE implementations.