litert-model-equivalence-test

@google-ai-edge · 收录于 1 周前 · 上游提交 1 周前

Validates equivalence between LiteRT models (litert_lm) and PyTorch models (transformers). Use when you need to verify that an exported LiteRT model produces the same outputs as the original Hugging Face model. Supports multi-turn conversations and custom prompts.

场景模型导出模型验证

适合你，如果需要在导出 LiteRT 模型后确保其与原始模型行为一致

/ 下载安装

https://github.com/google-ai-edge/litert-torch/tree/main/litert_torch/generative/export_hf/experimental/validation ↗

/ 通过 npx 安装 校验哈希

npx oh-my-skill add google-ai-edge/litert-torch/litert-model-equivalence-test

/ 通过 bash 安装

curl -fsSL https://oh-my-skill.com/install.sh | bash -s -- google-ai-edge/litert-torch/litert-model-equivalence-test

/ 已经装过?验证本机副本,不用重装

npx oh-my-skill verify google-ai-edge/litert-torch/litert-model-equivalence-test

安装目标可用 --agent / --scope 或 --to 明确指定;省略时只会在唯一已存在的 agent 目录上自动选择,零命中或多命中会停止并提示。content_hash 缺失或不一致均拒装。

1070GitHub stars

~609最小装载

~609含声明引用

~2.9K文本包总量

索引托管

怎么用

商店整理自技能原文 · 版本 677fe70 · 表述以原文为准

它做什么

运行一个测试脚本，比较 LiteRT 模型和原始 PyTorch 模型在相同输入下的输出是否一致。

什么时候触发

当你需要验证导出的 LiteRT 模型与 Hugging Face 原模型输出是否等价时。

装好后可以这样说

使用默认提示词运行等价测试。

运行多轮测试。

技能原文 SKILL.md作者撰写 · Apache-2.0 · 677fe70

LiteRT Model Equivalence Test

This skill provides instructions for running equivalence tests between LiteRT models and their PyTorch source models.

Usage

Use the equivalence_test script to compare the outputs of a Hugging Face model and its exported LiteRT version.

Running the Test

Run the test using bazel run from your workspace:

bazel run \
  //third_party/py/litert_torch/generative/export_hf/experimental/validation:equivalence_test \
  -- \
  --model_id={model_id} \
  [--prompt={prompt}] \
  [--prompt_file={prompt_file}] \
  [--max_new_tokens={max_new_tokens}] \
  [--max_num_tokens={max_num_tokens}] \
  [--work_dir={work_dir}] \
  [--externalize_embedder] \
  [--single_token_embedder] \
  [--split_cache] \
  [--backend={backend}]

Flags

--model_id: The Hugging Face model ID to validate (e.g., google/gemma-3-270m-it).
--prompt: Prompt to test. Specify multiple times for multi-turn conversations.
--prompt_file: Path to a file containing one (complex) prompt. Overrides --prompt.
--max_new_tokens: Maximum new tokens to generate per turn (default: 20).
--max_num_tokens: KV cache length for the model (default: 2048).
--work_dir: Base directory for model export. If not specified, a temporary directory under HOME is used.
--externalize_embedder: Externalize the embedder during export (default: False).
--single_token_embedder: Use single token embedder during export (default: False).
--split_cache: Split KV cache during export (default: False).
--backend: Hardware backend to use for LiteRT LM (cpu | npu, default: cpu).

Examples

Single-turn test with custom prompt:

bazel run \
  //third_party/py/litert_torch/generative/export_hf/experimental/validation:equivalence_test \
  -- \
  --model_id=google/gemma-3-270m-it \
  --prompt="What is the capital of France?"

Multi-turn test:

bazel run \
  //third_party/py/litert_torch/generative/export_hf/experimental/validation:equivalence_test \
  -- \
  --model_id=google/gemma-3-270m-it \
  --prompt="What's the capital of France?" \
  --prompt="How about Germany?"

Testing with a prompt file:

bazel run \
  //third_party/py/litert_torch/generative/export_hf/experimental/validation:equivalence_test \
  -- \
  --model_id=google/gemma-3-270m-it \
  --prompt_file=/path/to/prompts.txt

Testing with externalized embedder:

bazel run \
  //third_party/py/litert_torch/generative/export_hf/experimental/validation:equivalence_test \
  -- \
  --model_id=google/gemma-3-270m-it \
  --externalize_embedder \
  --single_token_embedder

Testing NPU export variant:

bazel run \
  //third_party/py/litert_torch/generative/export_hf/experimental/validation:equivalence_test \
  -- \
  --model_id=google/gemma-3-270m-it \
  --externalize_embedder \
  --split_cache \
  --backend=npu

按 Apache-2.0 许可原样转载,未经改动 · 在 GitHub 查看 →

登录即可评论;带「已验证安装」的,是发布者名下有本店的安装或持有记录。

…