Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iter: provide iterator example handling errors and resources #71788

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions src/iter/iter.go
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,23 @@ And then a client could delete boring values from the tree using:
}
}

# Iterators with complex datasource

Iterators encapsulating complex datasources offer value by separating
the iterator consumer from data-retrieval concerns in terms of databases,
networking and file systems. Intra-thread iteration over a function means
additional freedom in designing the iterator for concurrency, synchronization
and threading.

There are four needs on such iterators, referring to the below example:
1. Receive and maintain internal state: filename, errp, osFile
2. Provide iteration values and determine end of iteration: [LineReader.Lines]
3. Release resources upon end of iteration or panic: [LineReader.cleanup]
4. Propagate error conditions outside the for statement: errp

The below construct ensures faster stack allocation, as opposed to on the heap,
and features potentially reusable iterator-state encapsulated in struct.

[The Go Blog: Range Over Function Types]: https://go.dev/blog/range-functions
[range loop]: https://go.dev/ref/spec#For_range
*/
Expand Down
118 changes: 118 additions & 0 deletions src/iter/iter_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
// Copyright 2025 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

package iter_test

import (
"bufio"
"errors"
"fmt"
"iter"
"os"
"path/filepath"
)

func Example() {

// errorHandler prints error message and exits 1 on error
var err error
defer errorHandler(&err)

// create test file
var filename = filepath.Join(os.TempDir(), "test.txt")
if err = os.WriteFile(filename, []byte("one\ntwo\n"), 0o600); err != nil {
return
}

// iterate over lines from test.txt
// - first argument shows iterator allocated on the stack
// - second argument shows providing data to iterator
// - third argument shows receiving error from iterator
// - —
// - stack allocation is faster than heap allocation
// - LineReader is on stack even if NewLineReader is in another module
// - LineReader pointer receiver is more performant
for line := range NewLineReader(&LineReader{}, filename, &err).Lines {
fmt.Println("iterator line:", line)
}
// return here, err may be non-nil

// Output:
// iterator line: one
// iterator line: two
}

// LineReader provides an iterator reading a file line-by-line
type LineReader struct {
// the file lines are being read from
filename string
// a pointer to store occurring errors
errp *error
// the open file
osFile *os.File
}

// NewLineReader returns an iterator over the lines of a file
// - [LineReader.Lines] is iterator function
// - new-function provides LineReader encapsulation
func NewLineReader(fieldp *LineReader, filename string, errp *error) (lineReader *LineReader) {
if fieldp != nil {
lineReader = fieldp
lineReader.osFile = nil
} else {
lineReader = &LineReader{}
}
lineReader.filename = filename
lineReader.errp = errp

return
}

// Lines is the iterator providing text-lines from the file filename
// - defer cleanup ensures cleanup is executed on panic
// in Lines method or for block
// - cleanup updates *LineReader.errp
func (r *LineReader) Lines(yield func(line string) (keepGoing bool)) {
var err error
defer r.cleanup(&err)

if r.osFile, err = os.Open(r.filename); err != nil {
return // i/o error
}
var scanner = bufio.NewScanner(r.osFile)
for scanner.Scan() {
if !yield(scanner.Text()) {
return // iteration canceled by break or such
}
}
err = scanner.Err()
// reached end of file or error
}

// LineReader.Lines is iter.Seq string
var _ iter.Seq[string] = (&LineReader{}).Lines

// cleanup is invoked on iteration end or any panic
// - errp: possible error from Lines
func (r *LineReader) cleanup(errp *error) {
var err error
if r.osFile != nil {
err = r.osFile.Close()
}
if err != nil || *errp != nil {
// aggregate errors in order of occurrence
*r.errp = errors.Join(*r.errp, *errp, err)
}
}

// errorHandler prints error message and exits 1 on error
// - deferrable
func errorHandler(errp *error) {
var err = *errp
if err == nil {
return
}
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}