Skip to content

README.md

Latest commit

 

History

History
204 lines (148 loc) · 6.34 KB

README.md

File metadata and controls

204 lines (148 loc) · 6.34 KB

ComfyUI-AutoGuidance

A ComfyUI custom node implementing AutoGuidance from the paper:

Guiding a Diffusion Model with a Bad Version of Itself (Karras et al., 2024)
https://arxiv.org/abs/2406.02507

Repository: https://github.com/xmarre/ComfyUI-AutoGuidance

Reddit Thread with more info on usage: https://www.reddit.com/r/StableDiffusion/comments/1r2a7qo/release_comfyuiautoguidance_guide_the_model_with/

What this does

Classic CFG steers generation by contrasting conditional and unconditional predictions.
AutoGuidance adds a second model path (“bad model”) and guides relative to that weaker reference.

In practice, this gives you another control axis for balancing:

  • quality/faithfulness,
  • collapse/overcooking risk,
  • structure vs detail emphasis (via ramping).

Included nodes

This extension registers two nodes:

  • AutoGuidance CFG Guider (good+bad) (AutoGuidanceCFGGuider)
    Produces a GUIDER for use with SamplerCustomAdvanced.
  • AutoGuidance Detailer Hook (Impact Pack) (AutoGuidanceImpactDetailerHookProvider)
    Produces a DETAILER_HOOK for Impact Pack detailer workflows (including FaceDetailer).

Installation

Clone into your ComfyUI custom nodes directory and restart ComfyUI:

git clone https://github.com/xmarre/ComfyUI-AutoGuidance

No extra dependencies.

Basic wiring (SamplerCustomAdvanced)

  1. Load two models:
    • good_model
    • bad_model
  2. Build conditioning normally:
    • positive
    • negative
  3. Add AutoGuidance CFG Guider (good+bad).
  4. Connect its GUIDER output to SamplerCustomAdvanced guider input.

Impact Pack / FaceDetailer integration

Use AutoGuidance Detailer Hook (Impact Pack) when your detailer nodes accept a DETAILER_HOOK.

This injects AutoGuidance into detailer sampling passes without editing Impact Pack source files.

Important: dual-model mode must use truly distinct model instances

If you use:

  • swap_mode = dual_models_2x_vram

then ensure ComfyUI does not dedupe the two model loads into one shared instance.

Recommended setup

Make a real file copy of your checkpoint (same bytes, different filename), for example:

  • SDXL_base.safetensors
  • SDXL_base_BADCOPY.safetensors

Then:

  • Loader A (file 1) → good_model
  • Loader B (file 2) → bad_model

If both loaders point to the exact same path, ComfyUI may share/collapse model state and dual-mode behavior/performance can be incorrect.

Parameters (AutoGuidance CFG Guider)

Required

  • cfg
  • w_autoguide (effect is effectively off at 1.0; stronger above 1.0)
  • swap_mode
    • shared_safe_low_vram (safest/slowest)
    • shared_fast_extra_vram (faster shared swap, extra VRAM)
    • dual_models_2x_vram (fastest, highest VRAM, requires distinct instances)

Optional core controls

  • ag_combine_mode
    • sequential_delta (default; current behavior: CFG result + AG delta)
    • multi_guidance_paper (paper-style multi-guide extrapolation using good-cond, good-uncond, bad-cond)
    • In multi_guidance_paper, ag_delta_mode, ag_max_ratio, ramping, and project/reject controls are not used.
  • ag_delta_mode
    • bad_conditional (default, common starting point)
    • raw_delta
    • project_cfg
    • reject_cfg
  • ag_max_ratio (caps AutoGuidance push relative to CFG update magnitude)
  • ag_allow_negative
  • ag_ramp_mode
    • flat
    • detail_late
    • compose_early
    • mid_peak
  • ag_ramp_power
  • ag_ramp_floor
  • ag_post_cfg_mode
    • keep
    • apply_after
    • skip

Swap/debug controls

  • safe_force_clean_swap
  • uuid_only_noop
  • debug_swap
  • debug_metrics

Example setup (one working recipe)

This is one practical baseline, not a rule.

Models

  • Good side:
    • Base checkpoint + more fully-trained/specialized stack (e.g., 40-epoch character LoRA + DMD2/LCM, etc.)
  • Bad side options:
    • Base checkpoint + earlier/weaker checkpoint/LoRA (e.g., 10-epoch) with intentionally poor weighting
    • Base checkpoint + fewer adaptation modules
    • Base checkpoint only

Core idea: bad side should be meaningfully weaker/less specialized than good side.

Node settings example

  • cfg: 1.1
  • w_autoguide: 3.00
    • note: in multi_guidance_paper, values above 2.0 clamp to α=1.0 (pure AG share).
  • swap_mode: dual_models_2x_vram
  • ag_delta_mode: reject_cfg
  • ag_max_ratio: 0.75
  • ag_allow_negative: true
  • ag_ramp_mode: compose_early
  • ag_ramp_power: 2.0
  • ag_ramp_floor: 0.00
  • ag_post_cfg_mode: skip
  • safe_force_clean_swap: true
  • uuid_only_noop: false
  • debug_swap: true
  • debug_metrics: true

Practical tuning notes

  • In non-paper modes, increase w_autoguide above 1.0 to strengthen effect.
  • In multi_guidance_paper mode, weights are interpreted as:
    • α = clamp(w_autoguide - 1, 0, 1)
    • w_total = max(cfg - 1, 0)
    • w_cfg = (1 - α) * w_total
    • w_ag = α * w_total
    • CFG leg uses cfg_scale = 1 + w_cfg (including sampler CFG hooks).
    • final output = CFG(good, cfg_scale) + w_ag * (C - B) where C = good conditional and B = bad conditional.
  • To set paper interpolation directly (total g, mix α):
    • cfg = g
    • w_autoguide = 1 + α
    • ag_combine_mode = multi_guidance_paper
  • In multi_guidance_paper, ag_post_cfg_mode still works:
    • keep/skip: AG is mixed before post-CFG hooks.
    • apply_after: run post-CFG hooks on CFG-only output, then add AG delta.
  • In multi_guidance_paper, ag_delta_mode, ag_max_ratio, ag_ramp_mode, ag_ramp_power, and ag_ramp_floor are currently ignored.
  • compose_early tends to affect composition/structure earlier in denoise.
  • Try detail_late for a more late-step/detail-leaning influence.

VRAM and speed

AutoGuidance adds extra forward work versus plain CFG.

  • dual_models_2x_vram: fastest but highest VRAM and strict dual-instance requirement.
  • Shared modes: lower VRAM, slower due to swapping.

Suggested A/B evaluation

At fixed seed/steps, compare:

  • CFG-only vs CFG + AutoGuidance
  • different ag_ramp_mode
  • different ag_max_ratio caps
  • different ag_delta_mode

Feedback wanted

Useful community feedback includes:

  • what “bad model” definitions work best in real SD pipelines,
  • parameter combos that outperform standard CFG or NAG,
  • reproducible A/B examples with fixed seed + settings.

If this node helps, share workflows and side-by-side comparisons in issues/discussions.