Managing files

It is possible to work with files both as input to your job, and also as an output from your job.

Writing output files

In order to write to an output file, you can use open_output_file(), which returns a FileUploadContext to which you can write() data. It has to be used as a context manager, like in the example below.

Note

Using open_output_file(), does not currently allow you to specify a content-type. One will be inferred from the extension you set in the file name, using mimetypes.guess_type().

Example: Write to two output files with the file upload context

with JobClient.connect() as job:
    with job.open_output_file("my_output_file.txt") as text_file:
        text_file.write("a line of text\n")
        text_file.write("another line of text\n")
        text_file.write("a third line of text\n")

    with job.open_output_file("my_data.json") as json_file:
        json_file.write(json.dumps({"my_data": 123.456}))

Specifying content types

You may wish to specify the content type of the output file, as that can affect if the file will be downloaded when accessed in the browser, or if it will be shown in a new tab instead. It might also be useful, if you for some reason do not want to specifiy a file extension in the file name.

To do so, use the upload methods in the file_manager, for example upload_text().

Example: Upload a JSON file without an extension, manually setting the content type

with JobClient.connect() as job:
    data = json.dumps({"my_data": 123.456})
    job.file_manager.upload_text(
        "my_output_file_without_extension", data, content_type="application/json"
    )

Writing binary data

In order to write binary data, you also have to use the file_manager’s upload_text() method.

Example: Upload a binary file

with JobClient.connect() as job:
    data = b"my binary data: \xaa\xbb\xcc"
    job.file_manager.upload_bytes("binary.bin", data)

Reading files

In order to consume a file, you can access a parameter with the “file” type, and access the content of the file that was uploaded.