Tip

Hover your cursor over any code block in this tutorial, and a clipboard will appear. Click it to copy the contents of the code block.

Prerequisites

This tutorial does not assume that you have any familiarity with hydra-zen or Hydra. It does, however, assume that you are comfortable with using Python. If you are new to Python, consider consulting this resource for getting started with Python, and this tutorial on the essentials of Python.

Create and Launch a Basic Application with Hydra

In this tutorial we will create a basic application that we can configure and launch using Hydra, within a Python program (the next section of the tutorial will introduce a CLI). Although this project will be trivial, we will be introduced to the overarching design that is used for any Hydra-based project, as well to the core functionality provided by hydra-zen.

Getting Started

We will install hydra-zen and then we will create a Python script where we will create our application.

Installing hydra-zen

To install hydra-zen in your Python environment, run the following command in your terminal:

pip install hydra-zen

To verify that hydra-zen is installed as-expected, open a Python console and try importing hydra_zen.

>>> import hydra_zen

Creating a Script for our Application

Navigate to (or create) a directory where you are comfortable with files being written; running our code will leave behind some “artifacts” here. Create a new text file called my_app.py and open it in an editor.

Creating a Simple Hydra-Based Application

Let’s design our program to take in two (configurable) player names, and to log the players’ names to a text file called player_log.txt.

Our program will consist of two components, which comprises the structure of any Hydra-based application:

1. A “config”, which defines the configurable interface of our application.

2. A task function, which accepts the populated config, and whose body specifies the code that will be executed when our application is launched.

We will then use hydra_zen.launch(Config, task_function, overrides=<...>) to configure and run our task-function from within a python program.

Once we create this typical Hydra app, we will iterate on it and simplify things by using how hydra-zen’s hydra_zen.zen() wrapper.

Writing the Application

In my_app.py we’ll define a config and task function for this application. We will use make_config() to create the config. Write the following code in this file.

Contents of my_app.py:

from hydra_zen import make_config, instantiate

Config = make_config("player1", "player2")
# A dataclass-type with fields
#  - player1: Any
#  - player2: Any

def task_function(cfg):
    # cfg: Config
    obj = instantiate(cfg)

    # access the player names from the config
    p1 = obj.player1
    p2 = obj.player2

    # write the log with the names
    with open("player_log.txt", "w") as f:
        f.write("Game session log:\n")
        f.write(f"Player 1: {p1}\n" f"Player 2: {p2}")

    return p1, p2

Launching the Application

It’s time to run our application. Open a Python console – or a Jupyter notebook – in the same directory as my_app.py. First, we will import our config and our task function.

>>> from my_app import Config, task_function

We will also need to import hydra-zen’s launch() function.

>>> from hydra_zen import launch

Next, we will launch our application by providing the launch() function with: our config, our task function, and specific configured values for the player’s names. Here, we will use the names link and zelda for the names of player 1 and player 2, respectively. The version_base parameter controls some of the default behaviors of Hydra [1].

Launching our application

>>> job = launch(Config, task_function, overrides=["player1=link", "player2=zelda"], version_base="1.1")

Let’s inspect the completion status of this job by inspecting job.status; it should indicate COMPLETED.

>>> job.status
<JobStatus.COMPLETED: 1>

We can also directly access the value that is returned by our task-function.

>>> job.return_value
('link', 'zelda')

Warning

If you modify the contents of my_app.py, then you need to restart your Python console (or restart the kernel of your Jupyter notebook) and re-launch the application in order for these changes to take effect.

Inspecting the Results

Our application was designed to log the names of the players for that particular game session; let’s check that this log was written as-expected, and familiarize ourselves with the other files that Hydra writes when it launches an application.

First, we’ll create a simple Python function that will make it easy to print files in our Python console

>>> from pathlib import Path
>>> def print_file(x: Path):
...     with x.open("r") as f:
...         print(f.read())

By default, Hydra will create a directory called outputs, and will store the application’s outputs in a time-stamped subdirectory of the form outputs/${now:%Y-%m-%d}/${now:%H-%M-%S}. The particular subdirectory for our job is provided by job.working_dir.

>>> job_dir = Path(job.working_dir)  # type: ignore
>>> job_dir  # output will vary based on reader's date/time/OS
WindowsPath('outputs/2021-10-21/10-36-23')

The contents of this directory consists of: the log-file that our application wrote, a .hydra directory that details the configurations of this particular job, and a log-file written by Hydra.

>>> sorted(job_dir.glob("*"))
[WindowsPath('outputs/2021-10-21/10-36-23/.hydra'),
 WindowsPath('outputs/2021-10-21/10-36-23/player_log.txt'),
 WindowsPath('outputs/2021-10-21/10-36-23/zen_launch.log')]

Let’s verify that our application wrote the player-log as-expected.

>>> print_file(job_dir / "player_log.txt")
Game session log:
Player 1: link
Player 2: zelda

Great! The players’ names were recorded correctly.

The contents of the .hydra subdirectory is a collection of YAML files:

>>> sorted((job_dir / ".hydra").glob("*"))
[WindowsPath('outputs/2021-10-21/10-36-23/.hydra/config.yaml'),
 WindowsPath('outputs/2021-10-21/10-36-23/.hydra/hydra.yaml'),
 WindowsPath('outputs/2021-10-21/10-36-23/.hydra/overrides.yaml')]

To see the particular config that was passed to our task function for this job, we can inspect config.yaml.

>>> print_file(job_dir / ".hydra" / "config.yaml")
player1: link
player2: zelda

We successfully designed, configured, and launched an application using hydra-zen and Hydra! In the next tutorial, we will add a command line interface to this app.

In the final section, let’s see how we can simplify some of our code using special features provided by hydra-zen.

Simplifying Things with hydra_zen.zen()

We can simplify our task function, removing Hydra-specific logic from it, by using hydra_zen.zen().

Let’s update our task function in my_app.py to be a simple function whose signature determines the fields that will be extracted and instantiated from our config. Then we can use hydra_zen.builds(), instead of hydra_zen.make_config(), to generate our config based off of the task function’s signature.

Simplified contents of my_app.py:

from hydra_zen import builds

# note: type annotations are not required
def new_task_function(player1: str, player2: str):
    # write the log with the names
    with open("player_log.txt", "w") as f:
        f.write("Game session log:\n")
        f.write(f"Player 1: {player1}\n" f"Player 2: {player2}")

    return player1, player2

Config = builds(new_task_function, populate_full_signature=True)
# `builds` auto-populates the fields of our config based on the
# signature of `task_function`
#
# `Config` is dataclass-type whose fields are
#   - player1: str
#   - player2: str
#   - _target_: str = "my_app.task_function"

Wrapping this function as

Simplified contents of my_app.py:

from hydra_zen import zen

hydra_compat_task_fn = zen(task_function)

returns a Hydra-compatible task function, which accepts a single input config just like our original task function did. However, the zen wrapper adds the logic of extracting and instantiating the player1 and player2 fields from our config for us.

Now we will supply a zen-wrapped version of our task function to hydra_zen.launch(); all other aspects of launching our app and inspecting its results are unchanged.

Launching our application

>>> from my_app import Config, new_task_function
>>> from hydra_zen import zen, launch
>>> wrapped_fn = zen(new_task_function)
>>> job = launch(Config, wrapped_fn, overrides=["player1=link", "player2=zelda"], version_base="1.1")

See that we were able to launch the same app as before, but with some additional benefits:

• Our task function does not have any Hydra-specific logic and can be used for other purposes.

• We can utilize hydra_zen.builds() to auto-populate our config instead of hand-specifying the fields.

Reference Documentation

Want a deeper understanding of how hydra-zen and Hydra work? The following reference materials are especially relevant to this tutorial section.

• zen()

• builds()

• make_config()

• launch()

• Hydra’s version_base

Attention

Cleaning Up: To clean up after this tutorial, delete the outputs directory that Hydra created upon launching our application. You can find this in the same directory as your my_app.py file.

Footnotes

[1]

Specifically, we want to ensure that Hydra will change its working directory to its “jobdir” when we launch our app.