Ctrl+K

Changelog#

This is a record of all past hydra-zen releases and what went into them, in reverse chronological order. All previous releases should still be available on pip.

0.13.0rc1 - 2024-04-01#

Note

This is documentation for an pre-release version of hydra-zen. You can install this pre-release via pip install --pre hydra-zen

This release makes it possible to use pydantic’s runtime type-checking and parsing throughout your hydra-zen app 🚀🚀🚀

Check it out:

Adding pydantic parsing to a hydra-zen app#
from hydra_zen import store
from hydra_zen.third_party.pydantic import pydantic_parser

from pathlib import Path
from pydantic import PositiveInt


def main(path: Path, age: PositiveInt = 10):
    assert isinstance(path, Path)
    print(f"{path=!r} {age=!r}")


if __name__ == "__main__":
    from hydra_zen import zen

    store(main)
    store.add_to_hydra_store()

    zen(main,
       instantiation_wrapper=pydantic_parser,
    ).hydra_main(
       config_name="main",
       config_path=None,
       version_base="1.3",
    )

In a vanilla hydra-zen app we would not be able to create a Path instance from the CLI, nor would the PositiveInt annotation provide any sort of runtime constraint. But by specifying instantiation_wrapper=pydantic_parser, we can pass in a string-path from the CLI, and it will be parsed as a Path instance

Providing a string-path at the CLI gets parsed as a Path instance.#
$ python my_script.py path=./foo.txt
path=WindowsPath('foo.txt') age=10

and attempting to pass in a negative value for age will raise a ValidationError

The PositiveInt annotation enforces that age must be greater non-negative.#
$ python my_script.py path=./foo.txt age=-1
Traceback (most recent call last):
  ...
pydantic.error_wrappers.ValidationError: 1 validation error for MainConfig
age
  Input should be greater than 0 [type=greater_than, input_value=-11, input_type=int]

This also means that types like Literal, which are not supported by Hydra, can be leveraged in your apps.

Critically, this pydantic-mediation parsing occurs not just for main, but (recursively) for all config-targets that are instantiated in the process of calling main via zen. This means that even deeply-nested values are validated against their annotated types by pydantic.

More generally, you can add an arbitrary wrapping layer to all targets instantiated by zen, which is how the aforementioned pydantic parsing is implemented.

See pull request #666 for more details and to see how to completely disable Hydra’s type-checking in favor of pydantic’s.

New Features#

Bug Fixes#

  • Fixes incompatibility between zen-processing features (e.g. zen_meta) and iterative-build patterns. See pull request #638.

0.12.1 - 2024-01-21#

Minor improvements to the type annotations for hydra_zen.get_target() for compatibility with pyright 1.1.345+.

0.12.0 - 2023-12-07#

This release makes hydra-zen’s auto-config and type-refinement capabilities fully customizable and extensible, while still preserving static type-checking 🎉.

It does so by exposing a customizable class – hydra_zen.BuildsFn – with methods that can be overridden to modify the aforementioned auto-config and type-refinement capabilities. This class then exposes the classmethods just, builds, and make_config, which will leverage these customized capabilities.

Here is a stripped-down example.

Basic structure for adding custom auto-config support for a type.#
from hydra_zen import BuildsFn
from hydra_zen.typing import CustomConfigType

# We want builds/just/make_config to be able to automatically
# configure instances of `SomeType`
class SomeType:
    ...

# The type parameter provided to `BuildsFn[...]` updates the type
# annotations of the config-creation functions so that type-checkers
# know that `SomeType` is now supported.
class CustomBuilds(BuildsFn[CustomConfigType[SomeType]]):
    """
    - To customize type-refinement support, override `_sanitized_type`.
    - To customize auto-config support, override `_make_hydra_compatible`.
    - To customize the ability to resolve import paths, override `_get_obj_path`.
    """
    @classmethod
    def _make_hydra_compatible(
        cls, value, **kw
    ) -> SupportedPrimitive:
        # Take some value and return a Hydra-compatible config for it.
        if isinstance(value, SomeType):
            return cls.builds(SomeType)
        return super()._make_hydra_compatible(value, **kw)

# These config-creation functions now know how to automatically
# - and recursively - generate configs instances of `SomeType`
builds = CustomBuilds.builds
just = CustomBuilds.just
make_config = CustomBuilds.make_config

For more details and examples, see pull request #553.

New Features#

Documentation#

0.11.0 - 2023-07-13#

This release drops support for Python 3.7 (which reached its end of life), hydra-core 1.1 and for omegaconf 2.1; this enabled the removal of a lot of complex compatibility logic from hydra-zen’s source code, and to improve the behavior of zen().

Release Highlights#

hydra-zen now uses the trusted publishers method for publishing its build artifacts to PyPI via a protected GitHub Actions environment. In short, this improves security for users by further reducing the surface area by which malicious 3rd parties could attempt to upload tainted versions of hydra-zen. Note that hydra-zen has always abided by the most rigorous methods for secure publishing - we adopted the Trusted Publishers method one day after it became available ❤️.

builds() now has special behavior when it is passed a dataclass type that already possesses a _target_ field: _target_ is treated in a transitive way. E.g. build(builds(int)) is equivalent to builds(int). This enables a powerful “builder” pattern where configs can be populated in iterative, branching ways (with full type-checking support 😎).

Basic ‘builder’ pattern#
from hydra_zen import make_custom_builds_fn, instantiate

fbuilds = make_custom_builds_fn(populate_full_signature=True)

def foo(x, y, z): return x, y, z

base_cfg = fbuilds(foo, x=0, y=0, z=0)

c1 = fbuilds(base_cfg, x=1)
c2 = fbuilds(c1, y=2)
c3 = fbuilds(c2, z=3)

>>> [instantiate(c) for c in [c1, c2, c3]]
[(1, 0, 0), (1, 2, 0), (1, 2, 3)]

Improvements#

  • builds() now has a transitive property that enables iterative build patterns. See pull request #455

  • zen()’s instantiation phase has been improved so that dataclass objects and stdlib containers are returned instead of omegaconf objects. See pull request #448.

  • zen() can now be passed resolve_pre_call=False to defer the resolution of interpolated fields until after pre_call functions are called. See pull request #460.

  • Added support for NumPy 1.25.0

Bug Fixes#

Compatibility-Breaking Changes#

Most of these changes will not have any impact on users, based on download statistics and the particular code patterns impacted by the following changes.

  • Python 3.8 is now the earliest supported version of Python supported by hydra-zen

  • hydra-core 1.2.0 and omegaconf 2.2.1 are now the minimum supported versions of hydra-zen’s dependencies.

  • The auto-instantiation behavior of Zen and zen() have been updated so that nested dataclasses (nested within lists, dicts, and other dataclasses) will no longer be returned as omegaconf configs (see pull request #448).

  • just() not longer returns a frozen dataclass (see pull request #459).

  • Users that relied on patterns like builds(builds(...)) will find that pull request #455 has changed their behaviors. This new behavior can be disabled via builds(..., zen_convert={'flat_target': False})

  • zen()’s instantiation behavior was changed by pull request #448. See that PR for instructions on restoring the old behavior.

  • The signature-inspection logic of builds() has been modified to adopt and backport a fix made to inspect.signature() in Python 3.11.4. See pull request #497.

Documentation - 2023-03-11#

The following parts of the documentation underwent significant revisions:

0.10.2 - 2023-07-04#

This patch circumvents an upstream bug in pyright, which was causing pyright 1.1.305+ to report the return type of hydra_zen.make_custom_builds_fn() as “Unknown”.

0.10.1 - 2023-05-23#

Bans typing-extension v4.6.0 which has a breaking bug in it.

0.10.0 - 2023-03-05#

Release Highlights#

hydra_zen.launch() now accepts non-string values for its overrides, and it accepts a dictionary for improved ergonomics. Previously, users had to form Hydra CLI-compatible strings when calling launch, now overrides can be passed to the launch API as their native types.

Manually forming CLI-compatible overrides#
from hydra_zen import launch, instantiate, make_config

values_for_experiment = [random.uniform(0, 1) for i in range(10)]

jobs = launch(
   make_config(a=None, b=None),
   instantiate,
   overrides=[
         "a=1",
         "b=[1,2,3]",
         "+param=" + ",".join([str(i) for i in values_for_experiment])
   ],
   multirun=True
)
Specifying native Python values in launch API#
from hydra_zen import launch, instantiate, make_config, multirun, hydra_list

values_for_experiment = [random.uniform(0, 1) for i in range(10)]

jobs = launch(
   make_config(a=None, b=None),
   instantiate,
   overrides={
         "a": 1,
         "b": hydra_list([1, 2, 3]),
         "+param": multirun(values_for_experiment)
   },
   multirun=True
)

Improvements#

Documentation - 2023-01-22#

The following How-To guides were added:

0.9.1 - 2023-01-13#

Improvements#

Bug Fixes#

  • hydra_zen.zen()’s hydra_main method now handles string config_path entries properly (only for Hydra 1.3.0+). Previously Hydra could not find the path to the wrapped task function. hydra-zen will warn users that a string config_path is not supported via hydra_zen.zen() for Hydra 1.2 and earlier. See pull request #384.

0.9.0 - 2022-12-30#

Release Highlights#

This release introduces zen() and ZenStore, which enable hydra-zen users to eliminate Hydra-specific boilerplate code from their projects and to utilize new patterns and best practices for working with config stores.

The wrapper zen will automatically extract, resolve, and instantiate fields from a config in order to call the function that it has wrapped, thus saving the user from writing repetitive, hydra-specific boilerplate code in their function. Thus this wrapper enables users to replace the following Hydra-specific task function:

The “old school” way of designing a task function for a Hydra app#
import hydra
from hydra.utils import instantiate

@hydra.main(config_name="my_app", config_path=None, version_base="1.2")
def trainer_task_fn(cfg):
   model = instantiate(cfg.model)
   data = instantiate(cfg.data)
   partial_optim = instantiate(cfg.partial_optim)
   trainer = instantiate(cfg.trainer)

   optim = partial_optim(model.parameters())
   trainer(model, optim, data).fit(cfg.num_epochs)

if __name__ == "__main__":
   trainer_task_fn()

with a Hydra-agnostic task function that has an explicit signature:

Using zen to design a Hydra-agnostic task function#
# note: no Hydra or hydra-zen specific logic here
def trainer_task_fn(model, data, partial_optim, trainer, num_epochs):
   optim = partial_optim(model.parameters())
   trainer(model, optim, data).fit(num_epochs)

if __name__ == "__main__":
    from hydra_zen import zen

    # All config-field extraction & instantiation is automated/mediated by zen.
    # I.e. `zen` will extract & instantiate model, data, etc. from the input
    # config and pass it to `trainer_task_fn`
    zen(trainer_task_fn).hydra_main(config_name="my_app", config_path=None)

There are plenty more bells and whistles to zen(), refer to pull request #310 and its reference documentation for more details.

ZenStore is an abstraction over Hydra’s config store. It enables users to maintain multiple, isolated store instances before populating Hydra’s global config store. It also protects users from accidentally overwriting entries in Hydra’s global store. ZenStore possesses auto-config capabilities: it will automatically apply builds() and just() in intuitive ways on inputs to generate the stored configs.

Using hydra_zen.store auto-generate and store configs#
from hydra_zen import ZenStore
from torch.optim import Adam, AdamW, RMSprop

torch_store = ZenStore("torch_store")

# Specify defaults for storing entries (group=optim)
# and for generating configs (_partial_=True and lr=1e-3)
optim_store = torch_store(group="optim", zen_partial=True, lr=0.001)

# Automatically applies `builds(<obj>, zen_partial=True, lr=0.001)`
# to create and then store configs under the "optim" group
optim_store(Adam, name="adam", amsgrad=True)
optim_store(AdamW, name="adamw", betas=(0.1, 0.999))
optim_store(RMSprop, name="rmsprop")

torch_store.add_to_hydra_store()  # populate Hydra's global store

The store can also be populated using a decorator pattern [1], e.g.

Using hydra_zen.store as a decorator to auto-configure and store objects.#
from dataclasses import dataclass
from hydra_zen import store

profile_store = store(group="profile")

# Adds two store entries under the "profile" group of the store
# with configured defaults for `has_root`
@profile_store(name="admin", has_root=True)
@profile_store(name="basic", has_root=False)
@dataclass
class Profile:
    username: str
    schema: str
    has_root: bool


db_store = store(group="database")

# calls `builds(profile_database, [...])` under the hood and
# adds the config to the store under the "profile" group
@db_store(name="database")
@db_store(name="test_database", size=1)
def profile_database(size):
    ...

New Features#

  • hydra-zen now supports Python 3.11

  • Adds the zen() decorator (see pull request #310)

  • Adds the Zen decorator-class (see pull request #310)

  • Adds the ZenStore class (see pull request #331)

  • Adds hyda_zen.store, which is a pre-initialized instance of ZenStore (see pull request #331)

  • The option hydra_convert='object' is now supported by all of hydra-zen’s config-creation functions. So that an instantiated structured config can be converted to an instance of its backing dataclass. This feature was added by Hydra 1.3.0.

  • Adds auto-config support for torch.optim.optimizer.required so that the common pattern builds(<torch_optimizer_type>, populate_full_signature=True, zen_partial=True) works and exposes lr as a required configurable parameter. Thanks to @addisonklinke for requesting this in issue #257.

  • builds([…], zen_wrapper=…) can now accept a partial’d function as a wrapper.

Improvements#

Bug Fixes#

  • pull request #355 fixes an issue where the parameterized generic hydra_zen.typing.Partial[<...>] would return None for Python versions 3.9+. This prevented this annotation from being used by runtime type checkers.

Deprecations#

Compatibility-Breaking Changes#

  • Calling just() on a class-object or function will now return a frozen instance of a statically-defined dataclass. Previously it returned a dynamically-defined dataclass type. This change was made to improve pickle-compatibility and hashability of configs that are automatically generated by hydra-zen. This is unlikely to cause any issues for users.

  • Previously, any class decorated by hydrated_dataclass() would have a __module__ attribute set to typing. Now the class’s __module__ will reflect the module where its static definition resides. This enables pickle-compatibility (pull request #338). This is unlikely to cause any issues for users.

Mutable Default Values for Dataclasses#

Beginning in Python 3.11 dataclasses.dataclass() checks for mutable default values by assessing if an object possesses a __hash__ attribute. Previously it only considered set, dict, and list types to be mutable. Accordingly, dataclass instances are now considered to be mutable unless they are frozen or if unsafe_hash=True was specified.

Demonstrating change in mutability rules for dataclasses starting in Python 3.11#
from dataclasses import dataclass, field

@dataclass
class A:
   ...

@dataclass
class NoLongerValid:
   number: int = 1
   nested: A = A()  # will raise at runtime due to mutable default

@dataclass
class IsOK:
   number: int = 1
   nested: A = field(default_factory=lambda: A())

A ramification of the use of a default-factory in this example is that the field nested can only be accessed from an instance of IsOK, whereas non-factory defaults can be accessed from the dataclass type itself.

Default factories require access from dataclass instances; they cannot be accessed from the dataclass type.#
>>> hasattr(IsOK, "number")
True
>>> hasattr(IsOK, "nested")
False
>>> hasattr(IsOK(), "nested")
True

Because hydra-zen users frequently nest dataclasses, hydra-zen’s dataclass-creation functions (builds et al.) now specify unsafe_hash=True by default. Thus the following pattern is still valid:

The dataclasses produced by hydra-zen 0.9.0 are hashable by default so that existing patterns do not break in Python 3.11.#
from dataclasses import dataclass, field
from typing import Any

from hydra_zen import builds
from hydra_zen.typing import Builds

@dataclass
class Config:
    # This is still OK
    builds_dict: Builds[type[dict[Any, Any]]] = builds(dict)()

That being said, hydra-zen will now treat dataclass instances whose __hash__ attribute is None as mutable – regardless of the Python version – in order to ensure consistent behaviors across all supported Python versions. Thus the following pattern will now break

@dataclass
class A:
    ...

Conf = builds(dict, y=A(), zen_convert={'dataclass': False})

Conf.y  # this will raise in hydra_zen 0.9.0+
Conf().y  # this is OK

In general it is recommended that config fields be accessed from dataclass instances, not types. This will avoid all such default value/factory issues.

0.8.0 - 2022-09-13#

Release Highlights#

This release adds auto-config support for dataclass types and instances, including pydantic datclasses. Thus one can now include in a structured config type-annotations and default values that are not natively supported by Hydra, and then use builds() and/or just() to create a Hydra-compatible intermediate .

Consider the following dataclass; neither the type-annotation for reduction_fn nor its default values are supported by Hydra/omegaconf, and thus it cannot be serialized to a yaml file nor used in a Hydra config.

A dataclass that cannot be used natively within a Hydra app as a structured config.#
from typing import Callable, Sequence
from dataclasses import dataclass

@dataclass
class Bar:
   reduce_fn: Callable[[Sequence[float]], float] = sum  # <- not compat w/ Hydra

With the release of hydra-zen 0.8.0, we can now use just() to automatically create a Hydra-compatible config that, when instantiated, returns Bar():

Using just() to create a Hydra-compatible structured config#
>>> from hydra_zen import builds, just, instantiate, to_yaml
>>> just_bar = just(Bar())

>>> print(to_yaml(just_bar))
_target_: __main__.Bar
reduce_fn:
  _target_: hydra_zen.funcs.get_obj
  path: builtins.sum

>>> instantiate(just_bar)  # returns Bar()
Bar(reduce_fn=<built-in function sum>)

This auto-conversion process works recursively as well

Demonstrating recursive auto-conversion of dataclasses.#
>>> from statistics import mean
>>> @dataclass
... class Foo:
...     bar: Bar

>>> foobar = Foo(Bar(reduce_fn=mean))
>>> instantiate(just(foobar))
Foo(bar=Bar(reduce_fn=<function mean at 0x000001F224640310>))
>>> instantiate(builds(Foo, bar=Bar(sum)))
Foo(bar=Bar(reduce_fn=<built-in function sum>))

Thus we can include these Hydra-compatible intermediates in our Hydra config or config store, and then use instantiate() to create the desired dataclass instances of Bar() and Foo(Bar(mean)) within our app’s task function. Note that this functionality works with pydantic dataclasses as well, which enables us to leverage enhanced runtime value and type-checking.

Big thanks to Jasha10 for proposing and prototyping the crux of this new capability.

Compatibility-Breaking Changes#

This release drops support for Python 3.6. If you require Python 3.6, please restrict your hydra-zen installation dependency as hydra-zen<0.8.0.

Specifying make_custom_builds_fn([...], builds_bases=<...>) was deprecated in hydra-zen 0.7.0 (pull request #263). Accordingly, this option has now been removed from hydra_zen.make_custom_builds_fn().

The addition of auto-config support for dataclasses (pull request #301) changes the default behaviors of just() and builds(). Previously, all dataclass types and instances lacking a _target_ field would be left unprocessed by these functions, and omegaconf would convert dataclass types and instances alike to DictConfigs

hydra-zen < 0.8.0#
from hydra_zen import just, builds, to_yaml
from dataclasses import dataclass
from omegaconf import DictConfig

@dataclass
class A:
    x: int = 1

assert to_yaml(just(A)) == "x: 1\n"
assert to_yaml(just(A())) == "x: 1\n"
assert to_yaml(builds(dict, x=A)().x) == "x: 1\n"
assert to_yaml(builds(dict, x=A())().x) == "x: 1\n"

Now these objects will automatically be converted to corresponding targeted configs with the desired behavior under Hydra-instantiation:

hydra-zen >= 0.8.0#
from hydra_zen import just, builds, instantiate
from dataclasses import dataclass

@dataclass
class A:
    x: int = 1

assert instantiate(just(A)) is A
assert instantiate(builds(dict, x=A)().x) is A

assert str(just(A())()) == "Builds_A(_target_='__main__.A', x=1)"
assert str(builds(dict, x=A(), hydra_convert="all")()) == "Builds_dict(_target_='builtins.dict', _convert_='all', x=<class 'types.Builds_A'>)"

If you depended on the previous default behavior, you can recreate it by using the new zen-convert settings as so:

Restoring old default behavior#
from hydra_zen import just, make_custom_builds_fn
from functools import partial

just = partial(just, zen_convert={"dataclass": False})
builds = make_custom_builds_fn(zen_convert={"dataclass": False})

Improvements#

  • Adds auto-config support for dataclasses.dataclass (as highlighted above). (See pull request #301)

  • builds() no longer has restrictions on inheritance patterns involving PartialBuilds-type configs. (See pull request #290)

  • We now verify that basic use cases of our config-creation and instantiation functions type-check correctly via mypy. Previously, we had only assured type-checking behavior via pyright

  • Added ZenConvert typed dictionary to document new zen-convert options for builds(), just(), and make_config(). (See pull request #301)

  • Adds support for using builds(<target>, populate_full_signature=True) where <target> is a dataclass type that has a field with a default factory. (See pull request #299)

  • Adds auto-config support for pydantic.Field, improving hydra-zen’s ability to automatically construct configs that describe pydantic models and dataclasses. (See pull request #303)

  • Two new utility functions were added to the public API: is_partial_builds() and uses_zen_processing()

  • The automatic type refinement performed by builds() now has enhanced support for typing.Annotated, typing.NewType, and typing.TypeVarTuple. (See pull request #283)

  • Docs: Upgraded sphinx theme: dark mode is now available!

  • Docs: Re-enabled sphinx code auto-link

Support for New Hydra/OmegaConf Features

Bug Fixes#

  • builds() would raise a TypeError if it encountered a target whose signature contained the annotations ParamSpecArgs or ParamSpecKwargs. It can now sanitize these annotations properly. (See pull request #283)

0.7.1 - 2022-06-22#

Bug Fixes#

The validation that hydra-zen performs on hydra_defaults was overly restrictive. E.g. it would flag [{"some_group": None}] as invalid, even though null is permitted in Hydra’s default list syntax. This patch fixes this validation and updates the docs & annotations for hydra_defaults in builds() and make_config(). See pull request #287 for more details. Thanks to @mgrinshpon-doxel for the bug report.

0.7.0 - 2022-05-10#

New Features#

Support for defaults lists

Hydra’s defaults list field can be passed to builds() and make_config() via the new hydra_defaults argument. Basic runtime and static type-checking are performed on this field. See pull request #264 for more details and examples.

Improved functionality for types with Specialized hydra-zen support

just(), to_yaml(), and save_as_yaml() can directly operate on values of types with specialized support from hydra-zen; these values will automatically be converted to structured configs.

>>> from functools import partial
>>> from hydra_zen import to_yaml, just

>>> def f(x): return x**2
>>> partiald_f = partial(f, x=2)

>>> just(partiald_f)  # convert to structured config
PartialBuilds_f(_target_='__main__.f', _partial_=True, x=2)

>>> print(to_yaml(partiald_f))  # convert to yaml
_target_: __main__.f
_partial_: true
x: 2

See pull request #250 and pull request #259 for more details and examples.

Support for Upcoming Hydra/OmegaConf Features#

OmegaConf v2.2.0 is adding native support for the following types:

hydra-zen already provides support for these, but this version will defer to OmegaConf’s native support when possible. (See pull request #262)

OmegaConf v2.2.0 improves its type-checking, with added support for nested containers. Accordingly, hydra-zen’s automatic type refinement will no longer auto-broaden nested container types when OmegaConf v2.2.0+ is installed. (See pull request #261)

Hydra v1.2.0 is introducing a version_base parameter that can control default behaviors in hydra.run and hydra.initialize. Correspondingly, version_base is now exposed via launch. See pull request #273 for more details.

Deprecations#

pull request #263 deprecates the builds_bases argument in make_custom_builds(). It will be removed in hydra-zen v0.8.0. Users will need to specify builds_bases on a per-config basis via builds.

Bug Fixes#

  • hydra_zen.builds(<Child.class-method>) would create a config with the wrong target if <class-method> was defined on a parent of Child. See issue #265.

Improvements#

  • Fixed internal protocol of partial to be compatible with latest type-shed annotations.

  • Add missing annotation overloads for builds() and make_custom_builds()

  • Substantial source code reorganization

  • Improved pyright tests

0.6.0 - 2022-03-09#

This release focuses on improving hydra-zen’s type-annotations; it increases the degree to which IDEs and static-analysis tools can infer information about common hydra-zen code patterns.

It should be noted that hydra-zen leverages advanced typing features (e.g. recursive types) and that some type-checkers do not support these features yet. hydra-zen’s type annotations are validated by pyright. Thus we recommend that users leverage pyright and pyright-based language servers in their IDEs (e.g. using Pylance in VSCode) for the best experience.

(A note to VSCode users: make sure to set Type Checking Mode to basic in your IDE – it is disabled by default!)

Bug Fixes#

builds(<target>, builds_bases=(...)) now properly supports the case where a parent config introduces zen-processing features via inheritance. See pull request #236 for more details.

Improvements#

  • builds(<target>, populate_full_signature=True) now carries accurate type information about the target’s signature. Thus IDEs can now auto-complete the signature of the resulting structured config. See pull request #224 for examples and details.

  • Type-information is now dispatched by make_custom_builds_fn() for the common use-cases of populate_full_signature=True and zen_partial=True, respectively. See pull request #224 for examples and details.

  • hydra_zen.typing.ZenWrappers is now a publicly-available annotation. It reflects valid types for builds(..., zen_wrappers=<...>).

  • hydra-zen now has a pyright-verified type completeness score of 100%. Our CI now requires that this score does not drop below 100%. See pull request #226 for more details.

  • Improved compatibility with mypy (pull request #243)

Support for Upcoming Hydra Features#

Hydra 1.1.2 will introduce support for partial instantiation of targeted configs via the _partial_ field. builds(<target>, zen_partial=True) will now set the _partial_ field on the structured config rather than using hydra_zen.funcs.zen_processing to facilitate partial instantiation.

Hydra < 1.1.2#
>>> Conf = builds(dict, a=1, zen_partial=True)

>>> print(to_yaml(Conf))
_target_: hydra_zen.funcs.zen_processing
_zen_target: builtins.dict
_zen_partial: true
a: 1

>>> instantiate(Conf)
functools.partial(<class 'dict'>, a=1)
1.1.2 <= Hydra#
>>> Conf = builds(dict, a=1, zen_partial=True)

>>> print(to_yaml(Conf))
_target_: builtins.dict
_partial_: true
a: 1

>>> instantiate(Conf)
functools.partial(<class 'dict'>, a=1)

This change will only occur when one’s locally-installed version of hydra-core is 1.1.2 or higher. Structured configs and yamls that configure partial’d objects via hydra_zen.funcs.zen_processing are still valid and will instantiate in the same way as before. I.e. this is only a compatibility-breaking change for code that relied on the specific implementation details of the structured config produced by builds(<target>, zen_partial=True).

In accordance with this change, the definition of hydra_zen.typing.PartialBuilds has been changed; it now reflects a union of protocols: ZenPartialBuilds[T] | HydraPartialBuilds[T], both are which are now part of the public API of hydra_zen.typing.

(See pull request #186 and pull request #230 for additional details)

Compatibility-Breaking Changes#

hydra_zen.typing.PartialBuilds is no longer a runtime-checkable protocol. Code that used PartialBuilds in this way can be updated as follows:

hydra-zen < 0.6.0#
>>> from hydra_zen.typing import PartialBuilds

>>> Conf = builds(int, zen_partial=True)
>>> isinstance(Conf, PartialBuilds)
True
0.6.0 <= hydra-zen#
>>> from hydra_zen.typing import HydraPartialBuilds, ZenPartialBuilds

>>> Conf = builds(int, zen_partial=True)
>>> isinstance(Conf, (HydraPartialBuilds, ZenPartialBuilds))
True

0.5.0 - 2022-01-27#

This release primarily improves the ability of builds() to inspect and the signatures of its targets; thus its ability to both auto-generate and validate configs is improved. This includes automatic support for specifying “partial’d” objects – objects produced by functools.partial() – as configured values, and even as the target of builds().

New Features#

Improvements#

  • Fixed an edge case caused by an upstream bug in inspect.signature, which prevented builds() from accessing the appropriate signature for some target classes. This affected a couple of popular PyTorch classes, such as torch.utils.data.DataLoader and torch.utils.data.Dataset. See pull request #189 for examples.

  • When appropriate, builds(<target>, ...) will now consult <target>.__new__ to acquire the type-hints of the target’s signature. See pull request #189 for examples.

  • Fixed an edge case in the type-widening behavior in both builds() and make_config() where a Builds-like annotation would be widened to Any; this widening was too aggressive. See pull request #185 for examples.

  • Type widening will now be applied to configured fields where an interpolated variable – a string of form "${<var-name>}" – is specified. See issue #206 for rationale and examples.

  • Fixed incomplete annotations for builds(..., zen_wrappers=<..>). See pull request #180

Compatibility-Breaking Changes#

The deprecations introduced in v0.3.0 are now errors. Refer to those notes for details and for solutions for fixing stale code.

Notes#

It should be noted that the aforementioned improvements to builds() can change the interface to your app.

For instance, if you were configuring torch.utils.data.DataLoader, note the following difference in behavior:

import torch as tr
from hydra_zen import builds, to_yaml

# DataLoader was affected by a bug in `inspect.signature`
ConfLoader = builds(tr.utils.data.DataLoader, populate_full_signature=True)

Before 0.5.0:

>>> print(to_yaml(ConfLoader))  # builds could not access signature
_target_: torch.utils.data.dataloader.DataLoader

After:

>>> print(to_yaml(ConfLoader))
_target_: torch.utils.data.dataloader.DataLoader
dataset: ???
batch_size: 1
shuffle: false
sampler: null
batch_sampler: null
num_workers: 0
collate_fn: null
pin_memory: false
drop_last: false
timeout: 0.0
worker_init_fn: null
multiprocessing_context: null
generator: null
prefetch_factor: 2
persistent_workers: false

0.4.1 - 2021-12-06#

0.4.0 - 2021-12-05 introduced an undocumented, compatibility-breaking change to how hydra-zen treats enum.Enum values. This patch reverts that change.

0.4.0 - 2021-12-05#

This release makes improvements to the validation performed by hydra-zen’s config-creation functions. It also adds specialized support for types that are not natively supported by Hydra.

Also included is an important compatibility-breaking change and a downstream fix for an upstream bug in omegaconf (a library on which Hydra intimately depends). Thus it is highly recommended that users prioritize upgrading to hydra-zen v0.4.0.

New Features#

  • Strict runtime and static validation of configuration types. See pull request #163 for detailed descriptions and examples.

    hydra-zen’s config-creation functions now provide both strict runtime and static validation of the configured values that they are fed. Thus users will have a much easier time identifying and diagnosing bad configs, before launching a Hydra job.

  • Specialized support for additional configuration-value types. See pull request #163 for detailed descriptions and examples.

    Now values of types like complex and pathlib.Path can be specified directly in hydra-zen’s configuration functions, and hydra-zen will automatically construct nested configs for those values. Consult Configuration-Value Types Supported by Hydra and hydra-zen for a complete list of the additional types that are supported.

Compatibility-Breaking Changes#

We changed the behavior of builds() when populate_full_signature=True and one or more base-classes are specified for inheritance.

Previously, fields specified by the parent class would take priority over those that would be auto-populated. However, this behavior is unintuitive as populate_full_signature=True should behave identically as the case where one manually-specifies the arguments from a target’s signature. Thus we have changed the behavior accordingly. Please read more about it in pull request #174.

Bug Fixes#

The following bug was discovered in omegaconf <= 2.1.1: a config that specifies a mutable default value for a field, but inherits from a parent that provides a non-mutable value for that field, will instantiate with the parent’s field. Please read more about this issue, and our downstream fix for it, at pull request #172.

It is recommended that users upgrade to the latest version of omegaconf once it is released, which will likely include a proper upstream fix of the bug.

Other improvements#

hydra-zen will never be the first to import third-party libraries for which it provides specialized support (e.g., NumPy).

0.3.1 - 2021-11-13#

This release fixes a bug that was reported in issue #161. Prior to this patch, there was a bug in builds() where specifying populate_full_sig=True for a target that did not have **kwargs caused all user-specified zen-meta fields to be excluded from the resulting config.

0.3.0 - 2021-10-27#

This release adds many new features to hydra-zen, and is a big step towards v1.0.0. It also introduces some significant API changes, meaning that there are notable deprecations of expressions that were valid in v0.2.0.

Note

📚 We have completely rewritten our docs! The docs now follow the Diátaxis Framework for technical documentation authoring.

Join the Discussion 💬

The hydra-zen project now has a discussion board. Stop by and say “hi”!

New Features#

  • The introduction of builds(..., zen_wrappers=<>).

    This is an extremely powerful feature that enables one to modify the instantiation of a builds-config, by including wrappers in a target’s configuration. Read more about it here.

  • Rich support for runtime type-checking of configurations.

    Piggybacking off of the introduction of the zen_wrappers feature, hydra-zen now offers support for customized runtime type-checking. Presently, either of two type-checking libraries can be used: pydantic and beartype.

    The type-checking capabilities offered by validates_with_pydantic() and validates_with_beartype(), respectively, are both far more robust than those offered by Hydra.

  • A new, simplified method for creating a structured config, via make_config().

    This serves as a much more succinct way to create a dataclass, where specifying type-annotations is optional. Additionally, provided type-annotations and default values are automatically adapted to be made compatible with Hydra. Read more here.

  • make_custom_builds_fn(), which enables us to produce new “copies” of the builds() function, but with customized default-values.

  • get_target(), which is used to retrieve target-objects from structured configs. See pull request #94

  • builds(..., zen_meta=<dict>) users to attach “meta” fields to a targeted config, which will not be used by instantiate when building the target.

    A meta-field can be referenced via relative interpolation; this interpolation will be valid no matter where the configuration is utilized. See pull request #112.

Deprecations#

  • The use of both hydra_zen.experimental.hydra_run and hydra_zen.experimental.hydra_multirun are deprecated in favor of the the function launch().

  • Creating partial configurations with builds(..., hydra_partial=True) is now deprecated in favor of builds(..., zen_partial=True).

  • The first argument of builds() is now a positional-only argument. Code that specifies builds(target=<target>, ...) will now raise a deprecation warning; use builds(<target>, ...) instead. Previously, it was impossible to specify target as a keyword argument for the object being configured; now, e.g., builds(dict, target=1) will work. (See: #104).

  • All keyword arguments of the form zen_xx, hydra_xx, and _zen_xx are reserved by both builds() and make_config(), to ensure that future features introduced by Hydra and hydra-zen will not cause compatibility conflicts for users.

Additional Items#

0.2.0 - 2021-08-12#

This release:

Compatibility-Breaking Changes

  • The protocol hydra_zen.typing.DataClass is no longer available in the public namespace, as it is not intended for public use. To continue using this protocol, you can import it from hydra_zen.typing._implementations, but note that it is potentially subject to future changes or removal.

0.1.0 - 2021-08-04#

This is hydra-zen’s first stable release on PyPI! Although we have not yet released version v1.0.0, it should be noted that hydra-zen’s codebase is thoroughly tested. Its test suite makes keen use of the property-based testing library Hypothesis. Furthermore, 100% code coverage is enforced on all commits into main.

We plan to have an aggressive release schedule for compatibility-preserving patches of bug-fixes and quality-of-life improvements (e.g. improved type annotations). hydra-zen will maintain a wide window of compatibility with Hydra versions; we test against pre-releases of Hydra and will maintain compatibility with its future releases.

Footnotes#

On this page
Show Source