JSALT 2025 - Play Your Part Tutorials 😎

Below you will find a brief description of each of the 6 available tutorials in this repo:

• 1.single_llm_full_generation.ipynb

  Learn how to generate synthetic dialogues using a single LLM, including structured output, persona and scenario definitions, and saving/loading dialogues.

• 2.multi-agent_generation.ipynb

  Explore multi-agent dialogue generation by having two LLM-based agents role-play different personas and interact with each other to create realistic conversations.

• 3.multi-agent+orchestrator_generation.ipynb

  Introduces orchestration: controlling and injecting instructions into agents during dialogue, enabling fine-grained control and scenario-driven conversational flows.

• 4.dialog_analysis.ipynb

  Analyze and compare the structure and flow of generated synthetic dialogues versus real human dialogues using graph-based and qualitative methods.

• 5.mechanistic_interpretability.ipynb

  Dive into mechanistic interpretability: learn how to probe, visualize, and manipulate the internal activations of LLMs using TransformerLens, including hands-on experiments with activation steering and direction ablation.

• 6.binding_entities.ipynb

  Reproduce experiments from the paper "How do Language Models Bind Entities in Context?" to investigate how LLMs associate entities and attributes, and perform interventions in the activation space.

---

Resources

📑 Slides & Videos

• 📋 Google Slides (presentation)
• 🎥 Presentation Video (YouTube)

📓 Tutorial Notebooks & Videos

• 📔 Jupyter Notebooks (Open in Colab)
• 🎬 Tutorial Video (YouTube)

---

Getting started

⚠️ NOTE: If you have no access to compute, click on the "Open In Colab" badge above and ignore this README. If you do have access to compute with GPU, then you can proceed.

Tutorials have only one requirement: Apptainer. So, make sure you have installed Apptainer on your machine.

Once you have Apptainer running, the only thing needed is to download our Apptainer jsalt.sif file from HuggingFace by clicking here. This file contains everything we need inside.

Finally, let's check if our Apptainer is working: (if you have a cluster, don't forget to connect to a node with GPU first, e.g. salloc -A YOUR_USER -p gpu -t 08:00:00 --gpus rtx3090:1)

apptainer run --nv jsalt.sif

You should see you're now connected to Apptainer> in the terminal. If we do:

ls -lh

We see the apptainer by default does not have access to the host current direction (the content of the tutorials folder). Let's close it with Ctrl+D and call run but with the -H flag to set the apptainer home directorey to the host current directory:

cd to/your/tutorials

apptainer run -H $(pwd) --nv jsalt.sif

ls -lh

Now we should see all our tutorial files.

Note: alternatively to downloading the .sif file, you can built it from scratch, for instance using Slurm with:

sbatch apptainer_build.sbatch

You should first update the .sbatch file to match your credentials and computing, etc.

Let's try now try using gemma3-1b model with ollama, once connected to our apptainer:

(optional) change default ollama ~/.ollama/models folder to store the models to locally in our current tutorials folder:

export APPTAINERENV_OLLAMA_MODELS="$(pwd)/.ollama/models"

ollama serve &
ollama run qwen2.5:14b

>>> What model are you?

I’m Gemma, a large language model created by the Gemma team at Google DeepMind. I’m an open-weights model, which means I’m widely available for public use!

⚠️ In case you're getting x509: certificate signed by unknown authority when ollama tries to download the model via HTTPS, make sure to set the SSL_CERT_FILE environment variable to point to the certificate files, for example: (from outside the apptainer)

export APPTAINERENV_SSL_CERT_FILE=/usr/lib/ssl/certs/ca-certificates.crt

Pres Ctrl+D to close the chat. Let's try the following piece of Python code that uses LangChain to interact with our ollama models. Type python3 and paste the following code:

from langchain_ollama.chat_models import ChatOllama
llm = ChatOllama(model="qwen2.5:14b", temperature=0)
response = llm.invoke([
    ("system", "You are the best japanese translator in the world."
               "Translate the user sentence to casual japanese without explanations."),
    ("human", "I love JSALT workshop :)")
])
print(response.content)

JSALTワークショップ大好きです

Great! 😊 everything is working fine! we just need to download the dataset and that's it.

Dataset Preparation

cd datasets
git clone [email protected]:RasaHQ/STAR.git

Make sure you end up with the STAR dataset in datasets/STAR with dialogues and tasks folders inside.

⚠️ If you're getting an error message, clone STAR from outside the Apptainer inside the datasets folder.

Jupyter Notebook

Run the Jupyter server inside the apptainer first:

apptainer run -H $(pwd) --nv jsalt.sif

exec jupyter lab --no-browser --ip 0.0.0.0 --port 3300

And then copy the URL that was printed when running the Server including the token value. For instance, for the following log:

The url is the following:

http://127.0.0.1:3300/lab?token=7869cf958adb81af57fe17b4638518c6d18eb30777886f01

You can then paste this URL directly in your browser to get access to the notebooks.

Note: If you're running apptainer in a remote node, we will need to use SSH port forwaring/tunneling. In your host machine run:

ssh -N -L 3300:localhost:3300 NODE_NAME

You can connect using VS code too, instead of your browser, make sure you have the Jupyter extension installed. Then in VS Code, click on connect to server and paste the Jupyter server URL to connect.

We're going to use Ollama in our tutorials, and we can run Ollama server inside a cell by doing the following "trick" to allow running background processes within a cell:

import os
get_ipython().system = os.system  # <- little hack

!ollama serve &

Note: the first two lines are a simple hack to allow running background processes (e.g. "ollama serve &") inside the notebook.

---

Run my python script from outside the apptainer?

Another option is to use the Apptainer exec command to run the python script directly from outside the apptainer. Press Ctrl+D to close our apptainer. Now try running the test_chatollama.py as follows:

# in some environemnt D-Bus session bus (IPC) creates errors
# when running an apptainer instance, so, in case of errors,
# we can disable it with:
# unset DBUS_SESSION_BUS_ADDRESS

apptainer instance start --nv jsalt.sif jsalt
apptainer exec instance://jsalt ollama serve &

# run all the scripts we want to
apptainer exec instance://jsalt python3 test_chatollama.py

# once finished, we can stop the background instance
apptainer instance stop jsalt

JSALTワークショップ大好きです

Unable to use Apptainer?

Just make sure that you have ollama installed and the requerements.txt, that is, make sure you run (for instance inside a new conda environment):

curl -fsSL https://ollama.com/install.sh | sh
pip install -r requirements.txt

If you have access to GPU, make sure your environment have CUDA installed.

If your are running the tutorials on Google Colab, just click here.

That's it. Good luck! :)