Use Union and Literal for multiple choices

[rlouf]

For instance to choose between the numbers 1, 2 and 3:

model(prompt, Literal[1, 2, 3])

And to generate a json object that is valid to either the User or the Customer model:

model(prompt, Union[User, Customer])

[RobinPicard]

Right now Choice accepts for its definition either an Enum or a list[str], neither of which can be used directly.
Would do you think about:

• making Choice accept Enum, list[str], Union and Literal for its definition
• also letting the user use Enum, Union and Literal directly when calling the model/generator and at the top of the Generator function turning them into a Choice instance.

The advantage would be to give lots of options to the user, but still limit the number of different types the models have to handle in the format_input method of their ModelTypeAdapter. The downside is that the user would not be able to use the model.generate method directly (but do we want them to do that?).

[rlouf]

This is a v1.0 issue where they will be no distinct choice, json, format or regex functions and everything will be inferred from the output type specified by the user.

The equivalent of choice would be model(prompt, Literal[x, y, z]) or model(prompt, Enum). model(prompt, list[x]) now means "generate a list of X". model(prompt, Union[X, Y]) means "generate X or Y", which is more general than choice (unless it is model(prompt, Union[Literal[x], Literal[y]] which is equivalent to model(prompt, Literal[x, y]).

The way to think about it is "if this call were encapsulated in a function, what would this function's output type be?":

def foo(prompt: str) -> Literal[1, 2, 3]:
    return model(prompt, Literal[1, 2, 3])

def bar(prompt: str) -> List[Client]:
    return model(prompt, List[Client])

[rlouf]

Tangentially related, this makes me think that we need to make the output type of [X]Generation.__call__ match the output type specified by the user so mypy can look for type errors in a user's code.