ICAP Learning Agent

by parapple
5
4
3
2
1
Score: 44/100

Description

Learn inside your Obsidian notes — contextual explanations, recall with feedback and in-document requests, powered by your own Pi agent.

Reviews

No reviews yet.

Stats

80
stars
1,105
downloads
5
forks
12
days
1
days
1
days
4
total PRs
1
open PRs
0
closed PRs
3
merged PRs
3
total issues
1
open issues
2
closed issues
47
commits

Latest Version

a day ago

Changelog

  • The built-in agent is a trimmed version of Pi. It keeps Pi's sessions, models, logins, skills, context files, its read, edit, write and shell tools, and MCP, without the terminal parts. main.js is about 9 MB instead of 12.7 MB.

  • Use your own installed Pi. In Pi to run → Your own install, each task runs your Pi (pi --mode rpc, 1.5.0 or later) with all your extensions, skills and MCP servers. The plugin only looks for Pi and never installs it.

  • Tool scope and confirmation are two settings. Isolated gives the agent ICAP's tools and file tools inside the vault. Full adds the shell, your connectors and files outside the vault. The confirmation mode (Ask for each, Accept edits in the vault, Skip all confirmations) decides when the agent stops to ask. The agent's edits come to you as a change to review. The original of each changed file is kept for a month. A write to Obsidian's or Pi's own folders always asks.

  • Connectors. The settings show your MCP servers from ~/.pi/agent/mcp.json. You can turn them on or off, add or remove them. They are available in the full scope.

  • Request blocks take messages while the agent works. What you write waits in the block and is sent when the round ends, or reaches the agent at once with deliver now. You can paste, drop or choose images. You can regenerate or edit the last request, copy a reply, and see how full the model's context is.

  • Session notes. New session note makes a note where each finished round is added to a [!request] record (FORMAT.md 7).

  • Activity shows where a task's time went. A strip shows the wait for the model, its thinking, your turns and the tool calls. Each tool call shows its time and what it returned. A failed task says why. Copy the task's details puts all of it on the clipboard as text for a bug report.

  • Math renders. \( \) and \[ \] in questions, feedback and replies render as math. A writing line's hint with math renders too.

  • Fixed: lines after a kept callout were out of line. After keep in note, line numbers drifted and a click could select the line above. This is fixed.

  • Windows build fixes (#5, thanks @wzizary).

  • 内置 Agent 是精简版的 Pi。 保留了 Pi 的会话、模型、登录、技能、上下文文件,读取、编辑、写入和 shell 工具,以及 MCP,去掉了终端相关的部分。main.js 从 12.7 MB 减到约 9 MB。

  • 可以使用你自己安装的 Pi。 在 运行哪个 Pi → 你自己安装的 中,每个任务都会运行你的 Pi(pi --mode rpc,1.5.0 或更高版本),带上你所有的扩展、技能和 MCP 服务器。插件只会查找 Pi,从不安装。

  • 工具范围和确认方式是两个设置。 隔离 只给 Agent ICAP 的工具和库内的文件工具;完整 还会加上 shell、你的连接器和库外的文件。确认方式(逐项确认、自动接受编辑、跳过全部确认)决定 Agent 什么时候停下来问你。Agent 的修改会作为一处改动交给你审阅,每个被改文件的原件会保留一个月。写入 Obsidian 或 Pi 自己的文件夹时总会询问。

  • 连接器。 设置里会列出 ~/.pi/agent/mcp.json 中的 MCP 服务器,可以开关、添加或移除。它们在完整范围下可用。

  • Agent 工作时,请求块也能接收消息。 你写的内容会在块中等待,本轮结束时发出;也可以用 立即送达 马上交给 Agent。可以粘贴、拖入或选择图片;可以重新生成或修改最后一个请求、复制回复,并查看模型上下文的占用情况。

  • 会话笔记。 新建会话笔记 会创建一篇笔记,每一轮结束后都会追加到上方的 [!request] 记录中(FORMAT.md 7)。

  • 活动视图会显示任务的时间花在哪里。 一条时间条显示等待模型、模型思考、你的回合和工具调用;每个工具调用都会显示用时和返回内容。失败的任务会说明原因。复制任务的详情 会把这些内容以纯文本放到剪贴板,方便提交 bug 报告。

  • 数学公式可以正常渲染了。 提问、反馈和回复中的 \( \) 和 \[ \] 会渲染成公式;回复栏提示里的公式也会渲染。

  • 修复:保留的 callout 之后的行会错位。 点击 留在笔记里 后,行号会错位,点击时可能选中上一行。现在已修复。

  • Windows 构建修复(#5,感谢 @wzizary)。

README file from

Github

ICAP Learning Agent

An Obsidian plugin for learning inside your own notes. There is no chat panel: explanations, questions and feedback appear in the note you are reading, and whatever you keep is plain Markdown. Its agent is Pi, built into the plugin and connected to your own model provider; nothing else needs to be installed.

中文说明 · Obsidian 1.11.4+ on desktop, iPhone, iPad and Android · English and Chinese interface

Selecting "angle of attack" opens an explanation under the paragraph

Features

  • Explain. Select a word or passage; the explanation appears under it. Ask for deeper, an example, plainer, or a comparison, and keep it if it helped.
  • Recall. Explain a section from memory. The agent answers what you actually wrote and asks a follow-up.
  • Check. A few questions on a section or a whole note, with feedback on your answer.
  • Ask the agent here. A request block anywhere in a note. Point at files with [[ or @; use your Pi skills and commands with /; paste or drop images. Keep writing while the agent works: your words wait, or reach it at once with deliver now. Regenerate or edit your last request.
  • Session notes. New session note makes a conversation a note of its own: each finished round is written into it, so it syncs, links and searches like any note, and goes on from another device.
  • Documents. PDFs, EPUBs and Word files in the vault become readable trees: the agent opens a book's table of contents, reads only the chapters it needs, and cites pages you can click.
  • Interactive visuals. Sliders and diagrams you can drag, in a sandbox with no network.
  • Activity. The status bar shows what the agent is doing and lists recent tasks: their status, what they cost (tokens, cache hits, price), and every file they created or changed. Turn on Record tasks as notes and every task that does real work also gets a note with properties, listed in a native Base (ICAP/Activity.base) you can filter, group and change; a task's name leads back to where you asked.

A request block: notes made from a PDF, with the edit to an existing note proposed as a diff

Edits to your existing notes are proposed as a diff and applied only when you accept them. Notes stay readable without the plugin. The name comes from the ICAP framework (Chi & Wylie, 2014): learning deepens from Passive to Active, Constructive and Interactive.

Install

In Obsidian, open Settings → Community plugins → Browse, search for ICAP Learning Agent, then Install and Enable.

To install by hand instead, download main.js, manifest.json and styles.css from the latest release into <your vault>/.obsidian/plugins/icap-agent/, and enable the plugin there.

The first launch opens a guide note that walks through every interaction. Coming from ICPA (icpa-agent)? ICAP imports its settings on first start; then remove the old plugin.

Connect a model

Open Settings → ICAP Learning Agent → Model providers → Log in…, then sign in with a subscription (Claude Pro/Max, ChatGPT, GitHub Copilot…) or paste an API key. Pi stores it in ~/.pi/agent/auth.json, shared with the Pi CLI. A key exported in your shell profile also works.

For a service Pi does not list (Ollama, LM Studio, vLLM, a proxy, any OpenAI-compatible API), use Custom endpoint… next to it: a name, base URL, key and model ids. It is saved in Pi's ~/.pi/agent/models.json (%USERPROFILE%\.pi\agent\models.json on Windows), shared with the Pi CLI. A key written there is plain text; enter $NAME to read an environment variable instead.

On a phone or tablet. Pi needs Node, which Obsidian mobile does not have, so there ICAP runs its own agent on Pi's model libraries, with ICAP's tools only. Open Settings → ICAP Learning Agent → Model providers and add an API key for Anthropic, OpenAI, Google Gemini, OpenRouter, DeepSeek, Mistral, Groq or xAI. Keys are kept in Obsidian's keychain on that device only: never in the vault, never synced, so each device gets its own. Add endpoint… connects Ollama or LM Studio on your network, a proxy, or any OpenAI- or Anthropic-compatible API. When saved, ICAP checks once whether the server allows the app to stream from it. If it doesn't, replies go through Obsidian and arrive whole: you see "waiting for the complete reply" instead of text appearing, and stopping only stops the wait (the request is still billed). Subscription logins, Bedrock and Vertex are desktop only. The model you choose on mobile is separate from the desktop's.

What the agent may do. Two settings under Advanced decide it.

  • Tools. ICAP and the vault (isolated), the default: the agent asks you questions, creates notes, and reads and changes files inside the vault. Also shell and connectors (full) adds a shell, your connectors (the MCP servers in Pi's mcp.json), files outside the vault, and your Pi skills and prompt templates.
  • Confirmation. Ask for each, the default: every change shows its diff in the note first, and every shell command and connector call asks. Accept edits in the vault: changes inside the vault land at once and leave their diff in the block. Skip all confirmations: nothing asks; with the full tools that means anything on this computer, so ICAP shows a warning before turning it on.

Whatever you choose, a write to Obsidian's own folder or Pi's always asks, the original of each changed file is kept for a month in the plugin's folder, and every change is listed in Activity.

Your own Pi. If you have installed Pi (1.5.0 or later), Pi to run → Your own install makes each task start your pi as its own process instead of the one inside the plugin. Your extensions then run in Pi itself, as they do in a terminal. Logins, models, MCP servers and sessions are the same files either way. ICAP finds Pi on your PATH, or at the path you give under Pi location; it does not install or update it. This is how to use Pi extensions. Your own Pi acts by its own rules: the two settings above do not apply to it, and ICAP records what it changes without reviewing it.

Network, accounts and files outside the vault

  • Network. Requests, with the parts of your notes and documents the agent reads, go only to the model provider you choose. ICAP has no server, no telemetry and no analytics. Visuals have no network access. With the full tools, your connectors contact their own services, and so do the extensions of your own Pi.
  • Accounts. You need an account with a model provider, which bills you directly. ICAP needs no account.
  • Files outside the vault. ICAP reads Pi's folder ~/.pi/agent/ (settings, credentials, extensions, skills) and writes there when you sign in, add a custom endpoint, change the model or trust a project, as the Pi CLI does. On first use it runs your login shell once to pick up missing environment variables such as provider keys. With the isolated tools the agent reads only the outside folders you allow and writes only inside the vault; with the full tools it has a shell. With Your own install, ICAP starts the pi program on your computer for each task and talks to it over a socket only your user account can open; nothing listens on the network.
  • On mobile. Nothing outside the vault is read. API keys are in Obsidian's keychain on the device.
  • In the vault. Settings and sessions live in .obsidian/plugins/icap-agent/; so does the document cache (docs/). The vault gets task notes and Activity.base only if you turn on Record tasks as notes.

Troubleshooting and limitations

  • "No model has credentials": log in under Model providers. A key added to ~/.zshrc reaches Obsidian on its next launch; then press ↻.
  • main.js is about 9 MB: Pi for the desktop and the model libraries the mobile agent uses; a phone loads only the latter.
  • Upgrading from 1.3 or earlier with "Your Pi setup": if Pi 1.5.0 or later is installed, ICAP now runs it, and nothing changes. If not, ICAP keeps the built-in agent with the full tools and no confirmations, as before, but Pi extensions no longer load there; a notice says so once.
  • On mobile: no Pi setup (your skills, extensions, MCP servers, /compact), no file or shell tools, no folders outside the vault. Each turn of a request block replays the conversation to the model, since sessions are not cached there. If iOS suspends Obsidian while a reply is coming in, the task pauses; open it again and press continue.
  • Scanned PDFs without a text layer can't be read (no OCR).
  • An explanation you haven't kept disappears when the plugin reloads.

More

DESIGN.md (how it works) · FORMAT.md (what it writes into notes) · CONTRIBUTING.md · MIT