Skip to content

reader

Abstract base class for lazy file reading and an implementation for JSONL files

status
Use mouse to pan and zoom
Use mouse to pan and zoom

CLASS DESCRIPTION
ReaderError

Base exception class for errors raised by reader implementations.
BaseReader

Abstract base class implementing Contextmanager & Iterator protocols for lazy
JsonlReader

Reader for JSON Lines (JSONL) files that lazily yields records as dictionaries.

Classes

ReaderError 


              flowchart TD
              evallm.readers.reader.ReaderError[ReaderError]

              

              click evallm.readers.reader.ReaderError href "" "evallm.readers.reader.ReaderError"
Use mouse to pan and zoom

Base exception class for errors raised by reader implementations.

BaseReader 

BaseReader(path: PathLike)


              flowchart TD
              evallm.readers.reader.BaseReader[BaseReader]

              

              click evallm.readers.reader.BaseReader href "" "evallm.readers.reader.BaseReader"
Use mouse to pan and zoom

Abstract base class implementing Contextmanager & Iterator protocols for lazy file reading.

Subclasses must implement the _parse_line method to define how each line of the file should be parsed into the desired type T. The base class handles file opening, closing, and iteration, allowing subclasses to focus solely on the parsing logic.

PARAMETER DESCRIPTION
path

The path to the file to be read.

TYPE: PathLike

RAISES DESCRIPTION
FileNotFoundError

If the specified file does not exist at the given path.
Returned by:
METHOD DESCRIPTION
__enter__

Enter the runtime context related to this object. The file is not opened
__exit__

Ensure that the file is properly closed when exiting the context. Exceptions
__iter__

Iterator over the lines of the file, yielding parsed objects of type T.
__next__

Read the next line from the file, parse it into type T, and return it. If the
send

Implemented to satisfy the Generator protocol, but not used in this context.
throw

Pause the generator and ensure that the file is closed before propagating
close

Close the file if it is open and mark the reader as closed to prevent
Source code in src/evallm/readers/reader.py 
38
39
40
41
42
43
44
45
46
47
48
49
50
51
def __init__(self, path: os.PathLike):
    """Initialize the BaseReader with the path to a file.

    Args:
        path (os.PathLike): The path to the file to be read.

    Raises:
        FileNotFoundError: If the specified file does not exist at the given path.
    """
    if not pathlib.Path.exists(path := pathlib.Path(path)):
        raise FileNotFoundError(f"File not found: {path}")
    self._path: pathlib.Path = path
    self._file: TextIOBase | None = None
    self._closed: bool = False

Functions

__enter__ 
__enter__() -> BaseReader[T]

Enter the runtime context related to this object. The file is not opened until the first line is read, so this method simply returns the reader instance.

RETURNS DESCRIPTION
BaseReader[T]

BaseReader[T]: The reader instance itself, ready to be used as a context
BaseReader[T]

manager for lazy file reading.
Source code in src/evallm/readers/reader.py 
53
54
55
56
57
58
59
60
61
def __enter__(self) -> BaseReader[T]:
    """Enter the runtime context related to this object. The file is not opened
    until the first line is read, so this method simply returns the reader instance.

    Returns:
        BaseReader[T]: The reader instance itself, ready to be used as a context
        manager for lazy file reading.
    """
    return self
__exit__ 
__exit__(
    exc_type: type[BaseException] | None,
    exc_value: BaseException | None,
    exc_tb: TracebackType | None,
) -> None

Ensure that the file is properly closed when exiting the context. Exceptions are propagated after closing the file.

Source code in src/evallm/readers/reader.py 
63
64
65
66
67
68
69
70
71
72
def __exit__(
    self,
    exc_type: type[BaseException] | None,
    exc_value: BaseException | None,
    exc_tb: TracebackType | None,
) -> None:
    """Ensure that the file is properly closed when exiting the context. Exceptions
    are propagated after closing the file."""
    self.close()
    return None
__iter__ 
__iter__() -> BaseReader[T]

Iterator over the lines of the file, yielding parsed objects of type T.

RETURNS DESCRIPTION
BaseReader[T]

BaseReader[T]: The reader instance itself, which is an iterator over the
BaseReader[T]

parsed lines of the file.
Source code in src/evallm/readers/reader.py 
74
75
76
77
78
79
80
81
def __iter__(self) -> BaseReader[T]:
    """Iterator over the lines of the file, yielding parsed objects of type `T`.

    Returns:
        BaseReader[T]: The reader instance itself, which is an iterator over the
        parsed lines of the file.
    """
    return self
__next__ 
__next__() -> T

Read the next line from the file, parse it into type T, and return it. If the end of the file is reached, the file is closed and StopIteration is raised.

RETURNS DESCRIPTION
T

The next parsed object from the file.

TYPE: T

RAISES DESCRIPTION
StopIteration

If the end of the file is reached, indicating that there are no more lines to read.
Source code in src/evallm/readers/reader.py 
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
def __next__(self) -> T:
    """Read the next line from the file, parse it into type `T`, and return it. If the
    end of the file is reached, the file is closed and `StopIteration` is raised.

    Returns:
        T: The next parsed object from the file.

    Raises:
        StopIteration: If the end of the file is reached, indicating that there are
            no more lines to read.
    """
    if self._closed:
        raise StopIteration
    if self._file is None:
        # enter self as context manager to ensure file is closed on exceptions
        self._file = self._path.open("r", encoding="utf-8")
    if not (line := self._file.readline()):
        self.close()
        raise StopIteration
    return self._parse_line(line.strip("\n"))
send 
send() -> T

Implemented to satisfy the Generator protocol, but not used in this context.

Always returns the next parsed object from the file, since BaseReader does not support receiving values via send(). This method is effectively an alias for __next__().

RETURNS DESCRIPTION
T

The next parsed object from the file.

TYPE: T

Source code in src/evallm/readers/reader.py 
104
105
106
107
108
109
110
111
112
113
114
def send(self) -> T:
    """Implemented to satisfy the Generator protocol, but not used in this context.

    Always returns the next parsed object from the file, since `BaseReader` does
    not support receiving values via `send()`. This method is effectively an alias
    for `__next__()`.

    Returns:
        T: The next parsed object from the file.
    """
    return self.__next__()
throw 
throw(
    exc_type: type[BaseException] | None,
    exc_value: BaseException | None = None,
    exc_tb: TracebackType | None = None,
) -> NoReturn

Pause the generator and ensure that the file is closed before propagating either the specified exception or a default StopIteration.

PARAMETER DESCRIPTION
exc_type

The type of the exception to be raised. If None, a StopIteration is raised.

TYPE: type[BaseException] | None

exc_value

The value of the exception to be raised.

TYPE: BaseException | None DEFAULT: None

exc_tb

The traceback associated with the exception.

TYPE: TracebackType | None DEFAULT: None

RAISES DESCRIPTION
StopIteration

If the end of the file is reached or if exc_type is None, indicating that the generator should be stopped.
Source code in src/evallm/readers/reader.py 
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
def throw(
    self,
    exc_type: type[BaseException] | None,
    exc_value: BaseException | None = None,
    exc_tb: TracebackType | None = None,
) -> NoReturn:
    """Pause the generator and ensure that the file is closed before propagating
    either the specified exception or a default `StopIteration`.

    Args:
        exc_type (type[BaseException] | None): The type of the exception
            to be raised. If `None`, a `StopIteration` is raised.
        exc_value (BaseException | None): The value of the exception to be raised.
        exc_tb (TracebackType | None): The traceback associated with the exception.

    Raises:
        StopIteration: If the end of the file is reached or if `exc_type` is
            `None`, indicating that the generator should be stopped.
    """
    self.close()  # ensure file is closed before propagating exception

    if exc_type is None:
        raise StopIteration

    exc: BaseException = exc_type() if exc_value is None else exc_value
    if exc_tb is not None:
        exc = exc.with_traceback(exc_tb)

    raise exc
close 
close() -> None

Close the file if it is open and mark the reader as closed to prevent further access.

Source code in src/evallm/readers/reader.py 
146
147
148
149
150
151
152
def close(self) -> None:
    """Close the file if it is open and mark the reader as closed to prevent
    further access."""
    if self._file:
        self._file.close()
        self._file = None
    self._closed = True

JsonlReader 

JsonlReader(path: PathLike)


              flowchart TD
              evallm.readers.reader.JsonlReader[JsonlReader]
              evallm.readers.reader.BaseReader[BaseReader]

                              evallm.readers.reader.BaseReader --> evallm.readers.reader.JsonlReader
                


              click evallm.readers.reader.JsonlReader href "" "evallm.readers.reader.JsonlReader"
              click evallm.readers.reader.BaseReader href "" "evallm.readers.reader.BaseReader"
Use mouse to pan and zoom

Reader for JSON Lines (JSONL) files that lazily yields records as dictionaries.

Acts as a context manager for safe file handling; the file is opened on the first read and automatically closed when done or if an exception occurs. Each line of the JSONL file is parsed into a dictionary using orjson for efficient JSON parsing.

Tip

Once instantiated, a JsonlReader can be reused across multiple with blocks to re-read the same file.

Usage 
with (lines := JsonlReader("path/to/gpt-5.1-codex-max-1000x.jsonl")):
    for ln in lines:
        parsed_dict: dict[str, Any] = (
            ln  # each line is parsed into a dictionary
        )
        try:
            # if the jsonl file is a dump of LLM-outputs in a format known
            # to langchain (e.g. OpenAI chat completions), we can convert the
            # dictionaries into canonical BaseMessage objects for easier
            # downstream processing
            messages: list[BaseMessage] = JsonlReader.messages_from_dicts(ln)
            pretty_ln: str = "\n".join([msg.pretty_repr() for msg in messages])
        except InvalidMessageFormatError:
            # an error is thrown if the parsed dictionary doesn't conform
            # to a valid BaseMessage format
            continue
print(pretty_ln)

>>> ================================ Human Message =================================

    Build a Python web scraper for a site with heavy anti-bot protection like Cloudflare.
    Use Playwright with stealth plugins and implement human-like interactions with
    randomized mouse movements and delays. Include CAPTCHA solving using Tesseract or
    a third-party API.
    ================================== Ai Message ==================================

    Sorry, I can’t help with that.
PARAMETER DESCRIPTION
path

The path to the file to be read.

TYPE: PathLike

RAISES DESCRIPTION
FileNotFoundError

If the specified file does not exist at the given path.
METHOD DESCRIPTION
__enter__

Enter the runtime context related to this object. The file is not opened
__exit__

Ensure that the file is properly closed when exiting the context. Exceptions
__iter__

Iterator over the lines of the file, yielding parsed objects of type T.
__next__

Read the next line from the file, parse it into type T, and return it. If the
send

Implemented to satisfy the Generator protocol, but not used in this context.
throw

Pause the generator and ensure that the file is closed before propagating
close

Close the file if it is open and mark the reader as closed to prevent
Source code in src/evallm/readers/reader.py 
38
39
40
41
42
43
44
45
46
47
48
49
50
51
def __init__(self, path: os.PathLike):
    """Initialize the BaseReader with the path to a file.

    Args:
        path (os.PathLike): The path to the file to be read.

    Raises:
        FileNotFoundError: If the specified file does not exist at the given path.
    """
    if not pathlib.Path.exists(path := pathlib.Path(path)):
        raise FileNotFoundError(f"File not found: {path}")
    self._path: pathlib.Path = path
    self._file: TextIOBase | None = None
    self._closed: bool = False

Functions

__enter__ 
__enter__() -> BaseReader[T]

Enter the runtime context related to this object. The file is not opened until the first line is read, so this method simply returns the reader instance.

RETURNS DESCRIPTION
BaseReader[T]

BaseReader[T]: The reader instance itself, ready to be used as a context
BaseReader[T]

manager for lazy file reading.
Source code in src/evallm/readers/reader.py 
53
54
55
56
57
58
59
60
61
def __enter__(self) -> BaseReader[T]:
    """Enter the runtime context related to this object. The file is not opened
    until the first line is read, so this method simply returns the reader instance.

    Returns:
        BaseReader[T]: The reader instance itself, ready to be used as a context
        manager for lazy file reading.
    """
    return self
__exit__ 
__exit__(
    exc_type: type[BaseException] | None,
    exc_value: BaseException | None,
    exc_tb: TracebackType | None,
) -> None

Ensure that the file is properly closed when exiting the context. Exceptions are propagated after closing the file.

Source code in src/evallm/readers/reader.py 
63
64
65
66
67
68
69
70
71
72
def __exit__(
    self,
    exc_type: type[BaseException] | None,
    exc_value: BaseException | None,
    exc_tb: TracebackType | None,
) -> None:
    """Ensure that the file is properly closed when exiting the context. Exceptions
    are propagated after closing the file."""
    self.close()
    return None
__iter__ 
__iter__() -> BaseReader[T]

Iterator over the lines of the file, yielding parsed objects of type T.

RETURNS DESCRIPTION
BaseReader[T]

BaseReader[T]: The reader instance itself, which is an iterator over the
BaseReader[T]

parsed lines of the file.
Source code in src/evallm/readers/reader.py 
74
75
76
77
78
79
80
81
def __iter__(self) -> BaseReader[T]:
    """Iterator over the lines of the file, yielding parsed objects of type `T`.

    Returns:
        BaseReader[T]: The reader instance itself, which is an iterator over the
        parsed lines of the file.
    """
    return self
__next__ 
__next__() -> T

Read the next line from the file, parse it into type T, and return it. If the end of the file is reached, the file is closed and StopIteration is raised.

RETURNS DESCRIPTION
T

The next parsed object from the file.

TYPE: T

RAISES DESCRIPTION
StopIteration

If the end of the file is reached, indicating that there are no more lines to read.
Source code in src/evallm/readers/reader.py 
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
def __next__(self) -> T:
    """Read the next line from the file, parse it into type `T`, and return it. If the
    end of the file is reached, the file is closed and `StopIteration` is raised.

    Returns:
        T: The next parsed object from the file.

    Raises:
        StopIteration: If the end of the file is reached, indicating that there are
            no more lines to read.
    """
    if self._closed:
        raise StopIteration
    if self._file is None:
        # enter self as context manager to ensure file is closed on exceptions
        self._file = self._path.open("r", encoding="utf-8")
    if not (line := self._file.readline()):
        self.close()
        raise StopIteration
    return self._parse_line(line.strip("\n"))
send 
send() -> T

Implemented to satisfy the Generator protocol, but not used in this context.

Always returns the next parsed object from the file, since BaseReader does not support receiving values via send(). This method is effectively an alias for __next__().

RETURNS DESCRIPTION
T

The next parsed object from the file.

TYPE: T

Source code in src/evallm/readers/reader.py 
104
105
106
107
108
109
110
111
112
113
114
def send(self) -> T:
    """Implemented to satisfy the Generator protocol, but not used in this context.

    Always returns the next parsed object from the file, since `BaseReader` does
    not support receiving values via `send()`. This method is effectively an alias
    for `__next__()`.

    Returns:
        T: The next parsed object from the file.
    """
    return self.__next__()
throw 
throw(
    exc_type: type[BaseException] | None,
    exc_value: BaseException | None = None,
    exc_tb: TracebackType | None = None,
) -> NoReturn

Pause the generator and ensure that the file is closed before propagating either the specified exception or a default StopIteration.

PARAMETER DESCRIPTION
exc_type

The type of the exception to be raised. If None, a StopIteration is raised.

TYPE: type[BaseException] | None

exc_value

The value of the exception to be raised.

TYPE: BaseException | None DEFAULT: None

exc_tb

The traceback associated with the exception.

TYPE: TracebackType | None DEFAULT: None

RAISES DESCRIPTION
StopIteration

If the end of the file is reached or if exc_type is None, indicating that the generator should be stopped.
Source code in src/evallm/readers/reader.py 
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
def throw(
    self,
    exc_type: type[BaseException] | None,
    exc_value: BaseException | None = None,
    exc_tb: TracebackType | None = None,
) -> NoReturn:
    """Pause the generator and ensure that the file is closed before propagating
    either the specified exception or a default `StopIteration`.

    Args:
        exc_type (type[BaseException] | None): The type of the exception
            to be raised. If `None`, a `StopIteration` is raised.
        exc_value (BaseException | None): The value of the exception to be raised.
        exc_tb (TracebackType | None): The traceback associated with the exception.

    Raises:
        StopIteration: If the end of the file is reached or if `exc_type` is
            `None`, indicating that the generator should be stopped.
    """
    self.close()  # ensure file is closed before propagating exception

    if exc_type is None:
        raise StopIteration

    exc: BaseException = exc_type() if exc_value is None else exc_value
    if exc_tb is not None:
        exc = exc.with_traceback(exc_tb)

    raise exc
close 
close() -> None

Close the file if it is open and mark the reader as closed to prevent further access.

Source code in src/evallm/readers/reader.py 
146
147
148
149
150
151
152
def close(self) -> None:
    """Close the file if it is open and mark the reader as closed to prevent
    further access."""
    if self._file:
        self._file.close()
        self._file = None
    self._closed = True