10/18/2023

Go Generics in Witness

Author: Mikhail Swift

Go Generics
If you work with Go, you've undoubtedly heard a lot of buzz about generics or the notorious lack of them until Go 1.18 was released in March 2022. If you go digging, you can find plenty of posts and articles about why Go shouldn't have implemented generics or how generics should have been included from day 1. Generics have been one of the most significant changes Go has undergone, second to Go Modules. Yet if you work with Go day-to-day, there's a high chance you've yet to use them or even encountered code that uses them a year and a half later.
Outside of some toy libraries I wrote to play with Go's new generics, I had a use case in mind.

The Background

We started Witness with a few goals in mind, the most important of which was to enable developers to create and experiment with new attestations with little friction. To do so, we came up with the idea of Attestors, small single-purpose structs that collect some information about the supply chain or build environment when a user runs Witness.
For example, here's a snippet of what our Environment attestor looks like:
type Attestor interface {
    Attest(ctx *attestation.AttestationContext) error
}

type EnvironmentAttestor struct {
	OS        string            `json:"os"`
	Hostname  string            `json:"hostname"`
	Username  string            `json:"username"`
	Variables map[string]string `json:"variables,omitempty"`

	blockList map[string]struct{}
}

func (a *EnvironmentAttestor) Attest(ctx *attestation.AttestationContext) error {
	a.OS = runtime.GOOS
	a.Variables = make(map[string]string)

	if hostname, err := os.Hostname(); err == nil {
		a.Hostname = hostname
	}

	if user, err := user.Current(); err == nil {
		a.Username = user.Username
	}

	FilterEnvironmentArray(os.Environ(), a.blockList, func(key, val, _ string) {
		a.Variables[key] = val
	})

	return nil
}
Developers creating an Attestor must make a struct that implements the Attestor interface and provide it to the go-witness's Run function. This is simple enough when you're only in the context of go-witness: instantiating and configuring the attestors is a job for the library user. However, this quickly becomes a pain for anyone implementing a front end for the library. For instance, the Witness CLI needs to know what attestors are available, what options are available to configure them, and how to configure them. Since we may have many front ends that utilize the go-witness library, it becomes impractical to modify each one every time we add a new Attestor.
The solution we used was to create a registry of Attestors. The registry was simple enough: an unexported map whose key is the name of the Attestor, and value is a function that would return a new instance of the Attestor. Really, it's a simple factory pattern. Below is a simplified version of our first implementation of the registry.
var (
	attestorsByName = map[string]AttestorRegistration{}
)

type AttestorRegistration {
    Name string
    Factory func() Attestor
}

func RegisterAttestor(name string, factory func() Attestor) {
    attestorByName[name] = AttestorRegistration{
        Name: name,
        Factory: factory,
    }
}

func AttestorRegistrationByName(name string) (AttestorRegistration, bool) {
    registration, ok := attestorsByName[name]
    return registration, ok
}
We need to sort out how to communicate the configuration options available for each Attestor within the registry. This is where generics first enter the picture. First, we'll define a  ConfigOption struct. This struct will contain basic information such as a name, a human-readable description, a default value, and a setter function. We'll also add a type constraint named Configurable. This will include just a few primitives that we want to allow as options.
type Configurable interface {
	int | string | []string
}

type ConfigOption[T Configurable] struct {
	name        string
	description string
	defaultVal  T
	setter      func(Attestor, T) (Attestor, error)
}

func (co ConfigOption[T]) Name() string {
	return co.name
}

func (co ConfigOption[T]) DefaultVal() T {
	return co.defaultVal
}

func (co ConfigOption[T]) Description() string {
	return co.description
}

func (co ConfigOption[T]) Setter() func(Attestor, T) (Attestor, error) {
	return co.setter
}
And with a slight tweak to our registry, we can now store a slice of ConfigOptionss for each AttestorRegistration:
type AttestorRegistration {
    Name string
    Factory func() Attestor
    Options []ConfigOption
}
Uh-oh. You may have caught this by looking at it, but our changes here won't work, no matter how we tweak the new Options field. If we try the above, we're greeted with an error message:
cannot use generic type ConfigOption[T Configurable] without instantiation
If we pass in the type constraint, that doesn't make sense, and Go will still throw up on us. The problem is that Go doesn't have subtypes and, therefore, has no concept of covariance or contravariance. A slice of ConfigOption[int] is not a slice of ConfigOption[slice]; they cannot be held within the same slice. Slices can hold interfaces though...
type Configurer interface {
	Description() string
	Name() string
}
Hacky, I know. ConfigOption implements both functions, so we can now store ConfigOption[int]s and ConfigOption[string]s together in the same slice. We'll have to do some type assertions while using them, but it gets us moving forward.
type AttestorRegistration{
    Name string
    Factory func() Attestor
    Options []Configurer
}

func RegisterAttestor(name string, factory func() Attestor, opts ...Configurer) {
    attestorByName[name] = AttestorRegistration{
        Name: name,
        Factory: factory,
        Options: opts,
    }
}
Great! This works well, even if there are some rough edges. You can check out the final version that we used here.
Smooth sailing, until...

Signer Providers

We needed to add the ability to sign with Vault's PKI Secret Engine to Witness. We also recognized that we will have more Signer Providers in the future, and we realized we were facing the same issue we had with Attestors. Each time a Signer Provider was added, the developer would need to modify any front end of go-witness with new configuration flags.
So, let's make the Attestor registry generic. This is a trivial refactor. All we have to do is break out the Attestor registry to its own package, wrap the previous package level map into a generic struct, and we're off to the races.
type Registry[T any] struct {
	entriesByName map[string]Entry[T]
}

type FactoryFunc[T any] func() T

type Entry[T any] struct {
	Factory FactoryFunc[T]
	Name    string
	Options []Configurer
}

func (r Registry[T]) Register(name string, factoryFunc FactoryFunc[T], opts ...Configurer) {
	entry := Entry[T]{
		Name:    name,
		Factory: factoryFunc,
		Options: opts,
	}

	r.entriesByName[name] = entry
}
We'll also have to tweak the ConfigOption struct from before.
type ConfigOption[T any, TOption Option] struct {
	name        string
	description string
	defaultVal  TOption
	setter      func(T, TOption) (T, error)
}

func (co ConfigOption[T, TOption]) Name() string {
    return co.name
}

func (co ConfigOption[T, TOption]) DefaultVal() TOption {
	return co.defaultVal
}

func (co ConfigOption[T, TOption]) Description() string {
	return co.description
}

func (co ConfigOption[T, TOption]) Setter() func(T, TOption) (T, error) {
	return co.setter
}
And that's it! Easy!

Conclusion

Generics are a relatively new tool in the Go developer's toolbox. There's a time and place for them, and there are still some gotchas if you're used to a language like C# or Java (no generic methods, for instance).
Our use case for a generic factory isn't anything revolutionary, and there are ways we could have accomplished it without generics. Other solutions would have involved empty interfaces and allowed registries to have mixed types. Generics allowed us to do this more safely and understandably than before Go 1.18.