class Enumerator - Documentation for Ruby 4.0
Description: class Enumerator: A class which allows both internal and external iteration. An Enumerator can be created by the following methods.
Keywords: ruby,class,Enumerator

🔍

No recent searches

# class Enumerator

A class which allows both internal and external iteration.

An `Enumerator` can be created by the following methods.

- `Object#to_enum`

- `Object#enum_for`

- `Enumerator.new`

Most methods have two forms: a block form where the contents are evaluated for each item in the enumeration, and a non-block form which returns a new `Enumerator` wrapping the iteration.

    enumerator = %w(one two three).each
    puts enumerator.class # => Enumerator

    enumerator.each_with_object("foo") do |item, obj|
      puts "#{obj}: #{item}"
    end

    # foo: one
    # foo: two
    # foo: three

    enum_with_obj = enumerator.each_with_object("foo")
    puts enum_with_obj.class # => Enumerator

    enum_with_obj.each do |item, obj|
      puts "#{obj}: #{item}"
    end

    # foo: one
    # foo: two
    # foo: three

This allows you to chain Enumerators together. For example, you can map a list’s elements to strings containing the index and the element as a string via:

    puts %w[foo bar baz].map.with_index { |w, i| "#{i}:#{w}" }
    # => ["0:foo", "1:bar", "2:baz"]

## External Iteration

An `Enumerator` can also be used as an external iterator. For example, `Enumerator#next` returns the next value of the iterator or raises `StopIteration` if the `Enumerator` is at the end.

    e = [1,2,3].each   # returns an enumerator object.
    puts e.next   # => 1
    puts e.next   # => 2
    puts e.next   # => 3
    puts e.next   # raises StopIteration

`next`, `next_values`, `peek`, and `peek_values` are the only methods which use external iteration (and `Array#zip(Enumerable-not-Array)` which uses `next` internally).

These methods do not affect other internal enumeration methods, unless the underlying iteration method itself has side-effect, e.g. `IO#each_line`.

`FrozenError` will be raised if these methods are called against a frozen enumerator. Since `rewind` and `feed` also change state for external iteration, these methods may raise `FrozenError` too.

External iteration differs significantly from internal iteration due to using a Fiber:

- The `Fiber` adds some overhead compared to internal enumeration.

- The stacktrace will only include the stack from the `Enumerator`, not above.

- Fiber-local variables are not inherited inside the `Enumerator` `Fiber`, which instead starts with no Fiber-local variables.

- `Fiber` storage variables are inherited and are designed to handle `Enumerator` Fibers. Assigning to a `Fiber` storage variable only affects the current `Fiber`, so if you want to change state in the caller `Fiber` of the `Enumerator` `Fiber`, you need to use an extra indirection (e.g., use some object in the `Fiber` storage variable and mutate some ivar of it).

Concretely:

    Thread.current[:fiber_local] = 1
    Fiber[:storage_var] = 1
    e = Enumerator.new do |y|
      p Thread.current[:fiber_local] # for external iteration: nil, for internal iteration: 1
      p Fiber[:storage_var] # => 1, inherited
      Fiber[:storage_var] += 1
      y << 42
    end

    p e.next # => 42
    p Fiber[:storage_var] # => 1 (it ran in a different Fiber)

    e.each { p _1 }
    p Fiber[:storage_var] # => 2 (it ran in the same Fiber/"stack" as the current Fiber)

## Convert External Iteration to Internal Iteration

You can use an external iterator to implement an internal iterator as follows:

    def ext_each(e)
      while true
        begin
          vs = e.next_values
        rescue StopIteration
          return $!.result
        end
        y = yield(*vs)
        e.feed y
      end
    end

    o = Object.new

    def o.each
      puts yield
      puts yield(1)
      puts yield(1, 2)
      3
    end

    # use o.each as an internal iterator directly.
    puts o.each {|*x| puts x; [:b, *x] }
    # => [], [:b], [1], [:b, 1], [1, 2], [:b, 1, 2], 3

    # convert o.each to an external iterator for
    # implementing an internal iterator.
    puts ext_each(o.to_enum) {|*x| puts x; [:b, *x] }
    # => [], [:b], [1], [:b, 1], [1, 2], [:b, 1, 2], 3

### Public Class Methods

new(size = nil) { |yielder| ... }
Source

    static VALUE
    enumerator_initialize(int argc, VALUE *argv, VALUE obj)
    {
        VALUE iter = rb_block_proc();
        VALUE recv = generator_init(generator_allocate(rb_cGenerator), iter);
        VALUE arg0 = rb_check_arity(argc, 0, 1) ? argv[0] : Qnil;
        VALUE size = convert_to_feasible_size_value(arg0);

        return enumerator_init(obj, recv, sym_each, 0, 0, 0, size, false);
    }

Creates a new `Enumerator` object, which can be used as an `Enumerable`.

Iteration is defined by the given block, in which a “yielder” object, given as block parameter, can be used to yield a value by calling the `yield` method (aliased as `<<`):

    fib = Enumerator.new do |y|
      a = b = 1
      loop do
        y << a
        a, b = b, a + b
      end
    end

    fib.take(10) # => [1, 1, 2, 3, 5, 8, 13, 21, 34, 55]

The optional parameter can be used to specify how to calculate the size in a lazy fashion (see `Enumerator#size`). It can either be a value or a callable object.

produce(initial = nil, size: nil) { |prev| block } → enumerator
Source

    static VALUE
    enumerator_s_produce(int argc, VALUE *argv, VALUE klass)
    {
        VALUE init, producer, opts, size;
        ID keyword_ids[1];

        if (!rb_block_given_p()) rb_raise(rb_eArgError, "no block given");

        keyword_ids[0] = rb_intern("size");
        rb_scan_args_kw(RB_SCAN_ARGS_LAST_HASH_KEYWORDS, argc, argv, "01:", &init, &opts);
        rb_get_kwargs(opts, keyword_ids, 0, 1, &size);

        size = UNDEF_P(size) ? DBL2NUM(HUGE_VAL) : convert_to_feasible_size_value(size);

        if (argc == 0 || (argc == 1 && !NIL_P(opts))) {
            init = Qundef;
        }

        producer = producer_init(producer_allocate(rb_cEnumProducer), init, rb_block_proc(), size);

        return rb_enumeratorize_with_size_kw(producer, sym_each, 0, 0, producer_size, RB_NO_KEYWORDS);
    }

Creates an infinite enumerator from any block, just called over and over. The result of the previous iteration is passed to the next one. If `initial` is provided, it is passed to the first iteration, and becomes the first element of the enumerator; if it is not provided, the first iteration receives `nil`, and its result becomes the first element of the iterator.

Raising `StopIteration` from the block stops an iteration.

    Enumerator.produce(1, &:succ)   # => enumerator of 1, 2, 3, 4, ....

    Enumerator.produce { rand(10) } # => infinite random number sequence

    ancestors = Enumerator.produce(node) { |prev| node = prev.parent or raise StopIteration }
    enclosing_section = ancestors.find { |n| n.type == :section }

Using `::produce` together with `Enumerable` methods like `Enumerable#detect`, `Enumerable#slice_after`, `Enumerable#take_while` can provide Enumerator-based alternatives for `while` and `until` cycles:

    # Find next Tuesday
    require "date"
    Enumerator.produce(Date.today, &:succ).detect(&:tuesday?)

    # Simple lexer:
    require "strscan"
    scanner = StringScanner.new("7+38/6")
    PATTERN = %r{\d+|[-/+*]}
    Enumerator.produce { scanner.scan(PATTERN) }.slice_after { scanner.eos? }.first
    # => ["7", "+", "38", "/", "6"]

The optional `size` keyword argument specifies the size of the enumerator, which can be retrieved by `Enumerator#size`. It can be an integer, `Float::INFINITY`, a callable object (such as a lambda), or `nil` to indicate unknown size. When not specified, the size defaults to `Float::INFINITY`.

    # Infinite enumerator
    enum = Enumerator.produce(1, size: Float::INFINITY, &:succ)
    enum.size  # => Float::INFINITY

    # Finite enumerator with known/computable size
    abs_dir = File.expand_path("./baz") # => "/foo/bar/baz"
    traverser = Enumerator.produce(abs_dir, size: -> { abs_dir.count("/") + 1 }) {
      raise StopIteration if it == "/"
      File.dirname(it)
    }
    traverser.size  # => 4

    # Finite enumerator with unknown size
    calendar = Enumerator.produce(Date.today, size: nil) {
      it.monday? ? raise(StopIteration) : it + 1
    }
    calendar.size  # => nil

product(*enums) → enumerator
product(*enums) { |elts| ... } → enumerator
Source

    static VALUE
    enumerator_s_product(int argc, VALUE *argv, VALUE klass)
    {
        VALUE enums = Qnil, options = Qnil, block = Qnil;

        rb_scan_args(argc, argv, "*:&", &enums, &options, &block);

        if (!NIL_P(options) && !RHASH_EMPTY_P(options)) {
            rb_exc_raise(rb_keyword_error_new("unknown", rb_hash_keys(options)));
        }

        VALUE obj = enum_product_initialize(argc, argv, enum_product_allocate(rb_cEnumProduct));

        if (!NIL_P(block)) {
            enum_product_run(obj, block);
            return Qnil;
        }

        return obj;
    }

Generates a new enumerator object that generates a Cartesian product of given enumerable objects. This is equivalent to `Enumerator::Product.new`.

    e = Enumerator.product(1..3, [4, 5])
    e.to_a #=> [[1, 4], [1, 5], [2, 4], [2, 5], [3, 4], [3, 5]]
    e.size #=> 6

When a block is given, calls the block with each N-element array generated and returns `nil`.

### Public Instance Methods

e + enum → enumerator
Source

    static VALUE
    enumerator_plus(VALUE obj, VALUE eobj)
    {
        return new_enum_chain(rb_ary_new_from_args(2, obj, eobj));
    }

Returns an enumerator object generated from this enumerator and a given enumerable.

    e = (1..3).each + [4, 5]
    e.to_a #=> [1, 2, 3, 4, 5]

each { |elm| block } → obj
each → enum
each(*appending_args) { |elm| block } → obj
each(*appending_args) → an_enumerator
Source

    static VALUE
    enumerator_each(int argc, VALUE *argv, VALUE obj)
    {
        struct enumerator *e = enumerator_ptr(obj);

        if (argc > 0) {
            VALUE args = (e = enumerator_ptr(obj = rb_obj_dup(obj)))->args;
            if (args) {
    #if SIZEOF_INT < SIZEOF_LONG
                /* check int range overflow */
                rb_long2int(RARRAY_LEN(args) + argc);
    #endif
                args = rb_ary_dup(args);
                rb_ary_cat(args, argv, argc);
            }
            else {
                args = rb_ary_new4(argc, argv);
            }
            RB_OBJ_WRITE(obj, &e->args, args);
            e->size = Qnil;
            e->size_fn = 0;
        }
        if (!rb_block_given_p()) return obj;

        if (!lazy_precheck(e->procs)) return Qnil;

        return enumerator_block_call(obj, 0, obj);
    }

Iterates over the block according to how this `Enumerator` was constructed. If no block and no arguments are given, returns self.

### Examples

    "Hello, world!".scan(/\w+/)                     #=> ["Hello", "world"]
    "Hello, world!".to_enum(:scan, /\w+/).to_a      #=> ["Hello", "world"]
    "Hello, world!".to_enum(:scan).each(/\w+/).to_a #=> ["Hello", "world"]

    obj = Object.new

    def obj.each_arg(a, b=:b, *rest)
      yield a
      yield b
      yield rest
      :method_returned
    end

    enum = obj.to_enum :each_arg, :a, :x

    enum.each.to_a                  #=> [:a, :x, []]
    enum.each.equal?(enum)          #=> true
    enum.each { |elm| elm }         #=> :method_returned

    enum.each(:y, :z).to_a          #=> [:a, :x, [:y, :z]]
    enum.each(:y, :z).equal?(enum)  #=> false
    enum.each(:y, :z) { |elm| elm } #=> :method_returned

each_with_index {|(*args), idx| ... }
each_with_index
Source

    static VALUE
    enumerator_each_with_index(VALUE obj)
    {
        return enumerator_with_index(0, NULL, obj);
    }

Same as `Enumerator#with_index(0)`, i.e. there is no starting offset.

If no block is given, a new `Enumerator` is returned that includes the index.

each_with_object(obj) {|(*args), obj| ... }
each_with_object(obj)
Source

    static VALUE
    enumerator_with_object(VALUE obj, VALUE memo)
    {
        RETURN_SIZED_ENUMERATOR(obj, 1, &memo, enumerator_enum_size);
        enumerator_block_call(obj, enumerator_with_object_i, memo);

        return memo;
    }

Iterates the given block for each element with an arbitrary object, `obj`, and returns `obj`

If no block is given, returns a new `Enumerator`.

### Example

    to_three = Enumerator.new do |y|
      3.times do |x|
        y << x
      end
    end

    to_three_with_string = to_three.with_object("foo")
    to_three_with_string.each do |x,string|
      puts "#{string}: #{x}"
    end

    # => foo: 0
    # => foo: 1
    # => foo: 2

Also aliased as: with_object
feed obj → nil
Source

    static VALUE
    enumerator_feed(VALUE obj, VALUE v)
    {
        struct enumerator *e = enumerator_ptr(obj);

        rb_check_frozen(obj);

        if (!UNDEF_P(e->feedvalue)) {
            rb_raise(rb_eTypeError, "feed value already set");
        }
        RB_OBJ_WRITE(obj, &e->feedvalue, v);

        return Qnil;
    }

Sets the value to be returned by the next yield inside `e`.

If the value is not set, the yield returns nil.

This value is cleared after being yielded.

    # Array#map passes the array's elements to "yield" and collects the
    # results of "yield" as an array.
    # Following example shows that "next" returns the passed elements and
    # values passed to "feed" are collected as an array which can be
    # obtained by StopIteration#result.
    e = [1,2,3].map
    p e.next           #=> 1
    e.feed "a"
    p e.next           #=> 2
    e.feed "b"
    p e.next           #=> 3
    e.feed "c"
    begin
      e.next
    rescue StopIteration
      p $!.result      #=> ["a", "b", "c"]
    end

    o = Object.new
    def o.each
      x = yield         # (2) blocks
      p x               # (5) => "foo"
      x = yield         # (6) blocks
      p x               # (8) => nil
      x = yield         # (9) blocks
      p x               # not reached w/o another e.next
    end

    e = o.to_enum
    e.next              # (1)
    e.feed "foo"        # (3)
    e.next              # (4)
    e.next              # (7)
                        # (10)

inspect → string
Source

    static VALUE
    enumerator_inspect(VALUE obj)
    {
        return rb_exec_recursive(inspect_enumerator, obj, 0);
    }

Creates a printable version of e.

next → object
Source

    static VALUE
    enumerator_next(VALUE obj)
    {
        VALUE vs = enumerator_next_values(obj);
        return ary2sv(vs, 0);
    }

Returns the next object in the enumerator, and move the internal position forward. When the position reached at the end, `StopIteration` is raised.

### Example

    a = [1,2,3]
    e = a.to_enum
    p e.next   #=> 1
    p e.next   #=> 2
    p e.next   #=> 3
    p e.next   #raises StopIteration

See class-level notes about external iterators.

next_values → array
Source

    static VALUE
    enumerator_next_values(VALUE obj)
    {
        struct enumerator *e = enumerator_ptr(obj);
        VALUE vs;

        rb_check_frozen(obj);

        if (!UNDEF_P(e->lookahead)) {
            vs = e->lookahead;
            e->lookahead = Qundef;
            return vs;
        }

        return get_next_values(obj, e);
    }

Returns the next object as an array in the enumerator, and move the internal position forward. When the position reached at the end, `StopIteration` is raised.

See class-level notes about external iterators.

This method can be used to distinguish `yield` and `yield nil`.

### Example

    o = Object.new
    def o.each
      yield
      yield 1
      yield 1, 2
      yield nil
      yield [1, 2]
    end
    e = o.to_enum
    p e.next_values
    p e.next_values
    p e.next_values
    p e.next_values
    p e.next_values
    e = o.to_enum
    p e.next
    p e.next
    p e.next
    p e.next
    p e.next

    ## yield args       next_values      next
    #  yield            []               nil
    #  yield 1          [1]              1
    #  yield 1, 2       [1, 2]           [1, 2]
    #  yield nil        [nil]            nil
    #  yield [1, 2]     [[1, 2]]         [1, 2]

peek → object
Source

    static VALUE
    enumerator_peek(VALUE obj)
    {
        VALUE vs = enumerator_peek_values(obj);
        return ary2sv(vs, 1);
    }

Returns the next object in the enumerator, but doesn’t move the internal position forward. If the position is already at the end, `StopIteration` is raised.

See class-level notes about external iterators.

### Example

    a = [1,2,3]
    e = a.to_enum
    p e.next   #=> 1
    p e.peek   #=> 2
    p e.peek   #=> 2
    p e.peek   #=> 2
    p e.next   #=> 2
    p e.next   #=> 3
    p e.peek   #raises StopIteration

peek_values → array
Source

    static VALUE
    enumerator_peek_values_m(VALUE obj)
    {
        return rb_ary_dup(enumerator_peek_values(obj));
    }

Returns the next object as an array, similar to `Enumerator#next_values`, but doesn’t move the internal position forward. If the position is already at the end, `StopIteration` is raised.

See class-level notes about external iterators.

### Example

    o = Object.new
    def o.each
      yield
      yield 1
      yield 1, 2
    end
    e = o.to_enum
    p e.peek_values    #=> []
    e.next
    p e.peek_values    #=> [1]
    p e.peek_values    #=> [1]
    e.next
    p e.peek_values    #=> [1, 2]
    e.next
    p e.peek_values    # raises StopIteration

rewind → e
Source

    static VALUE
    enumerator_rewind(VALUE obj)
    {
        struct enumerator *e = enumerator_ptr(obj);

        rb_check_frozen(obj);

        rb_check_funcall(e->obj, id_rewind, 0, 0);

        e->fib = 0;
        e->dst = Qnil;
        e->lookahead = Qundef;
        e->feedvalue = Qundef;
        e->stop_exc = Qfalse;
        return obj;
    }

Rewinds the enumeration sequence to the beginning.

If the enclosed object responds to a “rewind” method, it is called.

size → int, Float::INFINITY or nil
Source

    static VALUE
    enumerator_size(VALUE obj)
    {
        struct enumerator *e = enumerator_ptr(obj);
        int argc = 0;
        const VALUE *argv = NULL;
        VALUE size;

        if (e->procs) {
            struct generator *g = generator_ptr(e->obj);
            VALUE receiver = rb_check_funcall(g->obj, id_size, 0, 0);
            long i = 0;

            for (i = 0; i < RARRAY_LEN(e->procs); i++) {
                VALUE proc = RARRAY_AREF(e->procs, i);
                struct proc_entry *entry = proc_entry_ptr(proc);
                lazyenum_size_func *size_fn = entry->fn->size;
                if (!size_fn) {
                    return Qnil;
                }
                receiver = (*size_fn)(proc, receiver);
            }
            return receiver;
        }

        if (e->size_fn) {
            return (*e->size_fn)(e->obj, e->args, obj);
        }
        if (e->args) {
            argc = (int)RARRAY_LEN(e->args);
            argv = RARRAY_CONST_PTR(e->args);
        }
        size = rb_check_funcall_kw(e->size, id_call, argc, argv, e->kw_splat);
        if (!UNDEF_P(size)) return size;
        return e->size;
    }

Returns the size of the enumerator, or `nil` if it can’t be calculated lazily.

    (1..100).to_a.permutation(4).size # => 94109400
    loop.size # => Float::INFINITY
    (1..100).drop_while.size # => nil

Note that enumerator size might be inaccurate, and should be rather treated as a hint. For example, there is no check that the size provided to `::new` is accurate:

    e = Enumerator.new(5) { |y| 2.times { y << it} }
    e.size # => 5
    e.to_a.size # => 2

Another example is an enumerator created by `::produce` without a `size` argument. Such enumerators return `Infinity` for size, but this is inaccurate if the passed block raises StopIteration:

    e = Enumerator.produce(1) { it + 1 }
    e.size # => Infinity

    e = Enumerator.produce(1) { it > 3 ? raise(StopIteration) : it + 1 }
    e.size # => Infinity
    e.to_a.size # => 4

with_index(offset = 0) {|(*args), idx| ... }
with_index(offset = 0)
Source

    static VALUE
    enumerator_with_index(int argc, VALUE *argv, VALUE obj)
    {
        VALUE memo;

        rb_check_arity(argc, 0, 1);
        RETURN_SIZED_ENUMERATOR(obj, argc, argv, enumerator_enum_size);
        memo = (!argc || NIL_P(memo = argv[0])) ? INT2FIX(0) : rb_to_int(memo);
        return enumerator_block_call(obj, enumerator_with_index_i, (VALUE)MEMO_NEW(memo, 0, 0));
    }

Iterates the given block for each element with an index, which starts from `offset`. If no block is given, returns a new `Enumerator` that includes the index, starting from `offset`

`offset`

  the starting index to use

with_object(obj) {|(*args), obj| ... }
with_object(obj)

Iterates the given block for each element with an arbitrary object, `obj`, and returns `obj`

If no block is given, returns a new `Enumerator`.

### Example

    to_three = Enumerator.new do |y|
      3.times do |x|
        y << x
      end
    end

    to_three_with_string = to_three.with_object("foo")
    to_three_with_string.each do |x,string|
      puts "#{string}: #{x}"
    end

    # => foo: 0
    # => foo: 1
    # => foo: 2

Alias for: each_with_object