Reading and writing data

Input variables

A components in a scenario can be configured with input variables, which are configured either with a reference to an input scenario, or with a simple static value.

Example: Accessing an input variable from a component

with JobClient.connect() as job:
    my_component = job.scenario.components["my_component"]
    input_variable = my_component.input_variables["my_input_variable"]
    ...

Reading data from inputs

All input variables in the scenario can be read using read_inputs(), which returns an iterable that returns tuples of time values and dictionaries of the data read. At the same time, value is also updated.

The row dictionaries returned by the iterable, are keyed by the dotted path of the input variable, which are the .-separated component names, and finally the input variable id.

Example: Iterate over all input variable values throughout the job runtime

with JobClient.connect() as job:
    result = 0.0
    for t, data in job.read_inputs(step=3600.0):
        my_variable_value = data["my_component_name.my_variable_id"]
        another_variable_value = data["another_component_name.another_variable_id"]
        result += my_variable_value / 2 + another_variable_value / 2

And using input variables it could be done like this:

Example: Accessing an input variable from a component

with JobClient.connect() as job:
    my_component = job.scenario.components["my_component"]
    input_variable = my_component.input_variables["my_input_variable"]
    result = 0.0
    for t, _ in job.read_inputs(step=3600.0):
        result += input_variable.value

Reading data from the current scenario

Sometimes reading data from the current scenario is useful, especially for auxillary jobs that are made to generate reports from the data written by another job (e.g. a simulation).

It is possible with the reader’s methods range() (for reading a simple range of data), and data_frame() for reading data into a pandas.DataFrame object.

Note

Using data_frame() requires pandas to be installed separately.

Example: Read the data series of the current scenario.

with JobClient.connect() as job:
    all_data = list(job.reader.range())
    data_frame = job.reader.data_frame()

Writing a timeseries of data

Writing data is quite simple. You can use either series() and write a whole series of data in one line, or you can use row() to write a single row of data. You can also combine the two, but it is important that the time value is always increasing, and is consistent.

Internally the data is buffered, and only actually uploaded once in a while to improve performance.

Example: Writing data both as single rows, and series

with JobClient.connect() as job:
    job.writer.row(
        job.time.start + timedelta(hours=1), {"tag1": 10.0, "tag2": 1.0}
    )
    job.writer.series(
        [job.time.start + timedelta(hours=h) for h in [2, 3, 4, 5]],
        {
            "tag1": [20.0, 30.0, 40.0, 50.0],
            "tag2": [1.3, 0.8, 10.0, -10.0]
        }
    )
    job.writer.row(
        job.time.start + timedelta(hours=6), {"tag1": 60.0, "tag2": -11.0}
    )

And instead of manually managing the time values, you can also iterate over range(), to get all valid time values within the job runtime.

Example: Writing rows of data using the runtime iterable

with JobClient.connect() as job:
    for t in job.time.range(step=timedelta(hour=1)):
        job.writer.row(t, {"tag1": compute_tag_1(t), "tag2": compute_tag_2(t)})

Combining reading and writing

Conveniently, it is possible to combine the reading and writing functions, so that it is easy to read data, transform it, and output the transformed values:

Example: Reading, transforming and then writing rows of data

with JobClient.connect() as job:
    for t, data in job.read_inputs(step=3600.0):
        my_variable_value = data["my_component_name.my_variable_id"]
        job.writer.row(t, {"with_added_noise": my_variable_value + random.random()})