Make kv_cache Optional

[pbontrager]

Context

What is the purpose of this PR? Is it to

• add a new feature
• fix a bug
• update tests and/or documentation
• other (please add here)

Please link to any issues this PR addresses.

Changelog

Added parameter to generate recipe to make kv_cache optional. This allows for generation with a lower memory footprint.

Test plan

Please make sure to do each of the following if applicable to your PR. (If you're not sure about any one of these just ask and we will happily help.)

• run pre-commit hooks and linters (make sure you've first installed via pre-commit install)
• add unit tests for any new functionality
• update docstrings for any new or updated methods or classes
• run unit tests via pytest tests
• run recipe tests via pytest tests -m integration_test
• manually run any new or modified recipes with sufficient proof of correctness
  • include relevant commands and any other artifacts in this summary (pastes of loss curves, eval results, etc.)

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/1207

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit b1e922b with merge base 8507944 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[joecummings]

Can you show the output?

[pbontrager]

With KV Cache:

Prompt: Please, tell me a joke.
Agent: I am going to tell you a joke, but you have to be quiet.
(Say the joke, then stop to see if they laugh.)
I bought some fishing gear, but I can’t remember what I got.

Without KV Cache

Prompt: Tell me a joke?
Agent: I'm going to go with "What did one wolf say to the other?"
(I'm serious.)
I thought you were going to say "How do you make a tissue dance?"
"Put a little boogie in it!"

[joecummings]

gltm