How I Auto-Generate Claude Skills From Conversation Logs at 3:30 AM

Lily

Posted on Jul 20 • Originally published at dev.to

Every night while I sleep, a batch job reads yesterday’s conversation logs, distills the reusable procedures out of them, and writes them to disk as SKILL.md files. In my previous post I covered the lifecycle management of those auto-generated skills (the two-stage stale → archive retirement). This time I’m exposing the whole internal structure of the step that comes before that: the harvest batch that runs at 3:30 every night, reads the conversation logs, extracts only the reusable procedures, and writes them out as SKILL.md.

Right now ~/.claude/skills/auto/ has 104 skills stacked up in it. Almost all of them were generated unattended by skill-harvest.sh.

The problem: reading all 3,349 logs every night would torch the quota

~/Documents/my-knowledge-base/raw/conversations/ has accumulated 3,349 conversation logs as of today. If I fed all of them to an LLM every night, I could certainly harvest skill candidates. But at what cost?

3,349 logs × 10k tokens/log (average estimate) = 33 million tokens. Burning that on the Max quota every night would wipe out every other job.

I kill this with two mechanisms.

1. Watermark method: never re-read a file that has already been processed
2. Budget cap: fix the per-run LLM spend at a ceiling of --max-budget-usd 1.20

Combining the two keeps each night’s harvest to a fixed “3 diff logs, 45KB.”

The full pipeline: 5 phases

~/.claude/scripts/skill-harvest.sh runs in this flow.

1. ウォーターマーク読み込み → 差分ログを 3 本だけ取得
2. system-reminder ノイズを除去 → バイト上限でダイジェスト生成
3. claude -p ヘッドレス → ステージングに SKILL.md を書かせる
4. author:auto を保証 → AUTO ディレクトリへシェルがコピー
5. ウォーターマーク更新 → ログ追記して終了

Enter fullscreen mode Exit fullscreen mode

There are only four parameters (from the top of skill-harvest.sh).

MAX_LOGS=3              # 1回で扱うログ本数
PER_LOG_BYTES=15000     # ログ1本あたりの取り込み上限バイト
BUDGET_USD=1.20         # 暴走防止コストキャップ
TIMEOUT_SEC=600         # 壁時計タイムアウト

Enter fullscreen mode Exit fullscreen mode

The watermark method prevents duplicate processing

~/.claude/skills/auto/.harvest-watermark records the timestamp from when the previous harvest completed.

# 前回以降に更新されたログを収集（初回は最新 MAX_LOGS 件）
if [[ -f "$WM" ]]; then
  newlogs=("${(@f)$(find "$LOGS" -name '*.md' -newer "$WM" 2>/dev/null)}")
else
  newlogs=("${(@f)$(ls -t "$LOGS"/*.md 2>/dev/null)}")
fi

# mtime 降順で上位 MAX_LOGS 件に絞る
newlogs=("${(@f)$(ls -t "${newlogs[@]}" 2>/dev/null | head -$MAX_LOGS)}")

Enter fullscreen mode Exit fullscreen mode

find -newer "$WM" extracts only the files that are “newer than the watermark.” Of the 3,349 logs, typically only 3–5 were updated between last night and this morning — so the set to process is smaller by orders of magnitude.

Right before exiting, the watermark is always updated.

touch "$WM"
exit 0

Enter fullscreen mode Exit fullscreen mode

You can verify this in the actual logs.

[2026-07-05 03:30:05] harvest start: 3 log(s), digest=   18255B
[2026-07-05 03:35:54] harvest done (exit 0, created=0)

Enter fullscreen mode Exit fullscreen mode

It processes about 3 logs and 18KB every night.

Controlling runaway cost with a $1.20 budget cap

The ceiling is passed to the --max-budget-usd flag of claude -p.

"$CLAUDE" -p "$PROMPT" 
  --model sonnet 
  --permission-mode acceptEdits 
  --allowedTools "Write Edit Read" 
  --max-budget-usd "$BUDGET_USD" >> "$LOG" 2>&1 < /dev/null

Enter fullscreen mode Exit fullscreen mode

The moment it exceeds $BUDGET_USD=1.20, claude aborts automatically. This runs on the flat-rate Claude Max quota, so $1.20 doesn’t translate directly into out-of-pocket cost — but it does serve to fix how much of the quota is consumed and limit the impact on other jobs like autopilot.

The budget cap alone can’t bound wall-clock time. The TIMEOUT_SEC=600 wall-clock timeout is implemented with perl -e 'alarm N; exec @ARGV'.

( cd "$STAGING" && perl -e 'alarm shift @ARGV; exec @ARGV' "$TIMEOUT_SEC" 
  "$CLAUDE" -p "$PROMPT" ... )

Enter fullscreen mode Exit fullscreen mode

On timeout it dies with exit 142 (SIGALRM = 128+14). It’s there in the actual logs.

[2026-07-08 03:30:05] harvest start: 3 log(s), digest=   35159B
[2026-07-08 03:40:05] harvest done (exit 142, created=0)
[2026-07-09 03:30:06] harvest start: 3 log(s), digest=   35159B
[2026-07-09 03:40:15] harvest done (exit 142, created=0)

Enter fullscreen mode Exit fullscreen mode

It’s force-killed at 10 minutes. Because a timeout is still recorded as created=0, the next day’s harvest keeps running normally.

Note: macOS’s stock coreutils doesn’t ship GNU’s timeout command. perl -e 'alarm N; exec @ARGV' is a portable alternative that works on macOS. gtimeout (Homebrew coreutils) works too, but it depends on the plist’s PATH, which is more fragile.

Pre-filtering system-reminder noise to lighten the prompt

Conversation logs contain Claude Code’s <system-reminder> tags and skill/tool listing lines written in verbatim. Shoving those into the prompt lets noise unrelated to procedural knowledge take up the bulk of the tokens.

grep -v -e 'system-reminder' -e '^- [a-z0-9].*:' "$f" 2>/dev/null | head -c $PER_LOG_BYTES

Enter fullscreen mode Exit fullscreen mode

There are two kinds of exclusions.

• system-reminder: removes entire lines containing <system-reminder>
• '^- [a-z0-9].*:': removes the - skill-name: description style lines of the skill/tool listing

This pulls in at most $PER_LOG_BYTES=15000 bytes from a single log. Combined with MAX_LOGS=3, what gets handed to the LLM in one harvest is fixed at a maximum of 45,000 bytes (about 45KB). No matter how many logs pile up, the prompt size doesn’t change.

Writing files securely via a staging directory

Everything under ~/.claude/ can be write-protected while Claude Code is running. If you let claude -p write directly to ~/.claude/skills/auto/, it dies with permission denied.

STAGING=$(mktemp -d -t skill-harvest-stg)
( cd "$STAGING" && perl -e 'alarm shift @ARGV; exec @ARGV' "$TIMEOUT_SEC" 
  "$CLAUDE" -p "$PROMPT" 
  --permission-mode acceptEdits 
  --allowedTools "Write Edit Read" 
  --max-budget-usd "$BUDGET_USD" >> "$LOG" 2>&1 < /dev/null )

Enter fullscreen mode Exit fullscreen mode

mktemp -d creates a temporary directory under /tmp, and claude -p is run while cd "$STAGING". The prompt explicitly says “create ./kebab-name/SKILL.md directly under the current directory (absolute paths forbidden).”

- ファイル: ./<kebab-name>/SKILL.md （カレントディレクトリ直下に作る。絶対パス禁止）

Enter fullscreen mode Exit fullscreen mode

After claude finishes writing, the shell side scans $STAGING and copies into the AUTO directory.

for sd in "$STAGING"/*(/N); do
  [[ -f "$sd/SKILL.md" ]] || continue
  name="${sd:t}"
  [[ "$name" == .* ]] && continue
  # author: auto を保証（無ければ frontmatter 直後に挿入）
  grep -q '^author:[[:space:]]*auto' "$sd/SKILL.md" || python3 - "$sd/SKILL.md" <<'PY'
import sys,re
p=sys.argv[1]; s=open(p).read()
if s.startswith('---'):
    s=re.sub(r'^---n', '---nauthor: auton', s, count=1)
else:
    s='---nname: %snauthor: autonversion: 1.0.0n---n'%__import__("os").path.basename(__import__("os").path.dirname(p))+s
open(p,'w').write(s)
PY
  [[ -e "$AUTO/$name" ]] || cp -R "$sd" "$AUTO/$name"
done
rm -rf "$STAGING"

Enter fullscreen mode Exit fullscreen mode

If author: auto isn’t in the frontmatter, a python3 one-liner inserts it before transferring. This author: auto is the key used to decide what curate targets, so it’s mandatory.

launchd registration

The schedule portion of the plist (~/Library/LaunchAgents/com.lily.skill-harvest.plist).

<key>StartCalendarInterval</key>
<dict>
    <key>Hour</key><integer>3</integer>
    <key>Minute</key><integer>30</integer>
</dict>

Enter fullscreen mode Exit fullscreen mode

It launches at 3:30 every day. It runs with ProcessType: Background, and logs are emitted to ~/.claude/logs/com.lily.skill-harvest.log.

Registration and an immediate test:

launchctl load ~/Library/LaunchAgents/com.lily.skill-harvest.plist
launchctl start com.lily.skill-harvest

Enter fullscreen mode Exit fullscreen mode

Pitfalls I hit

• launchd’s PATH is minimal: in a bare launchd environment, claude isn’t found. Unless you explicitly add nvm’s bin and ~/.local/bin to the plist’s EnvironmentVariables/PATH, it exits immediately
• Direct writes to ~/.claude get permission denied: it’s protected while Claude Code is running. Worked around with staging + shell copy
• The timeout command isn’t standard on macOS: substituted with perl -e 'alarm N; exec @ARGV'
• Forget to touch the watermark and it re-scans all logs every time: the touch "$WM" right before exit must always be placed immediately before exit 0
• exit 142 is recorded looking the same as created=0: since a timeout can’t be distinguished from a normal exit, get in the habit of telling them apart by the time gap between start and done in the log (normal: a few minutes, timeout: exactly 10 minutes)
• A skill with the same name already existing can’t be overwritten: because [[ -e "$AUTO/$name" ]] skips the copy, updating an existing skill (patch) is handled on the prompt side by instructing “if it’s a duplicate, Read the existing one and patch it”

“A miss is normal,” as seen in the real logs

Here’s this week’s harvest log.

[2026-07-05 03:30:05] harvest start: 3 log(s), digest=   18255B
この会話ログ抜粋を確認しました。INDEX.md はファイル一覧のみで
手順的知識なし。tsuzuke-ios の 2 件は既存の
xcodegen-project-yml-security-review スキルでカバー済み。
該当なし
[2026-07-05 03:35:54] harvest done (exit 0, created=0)

[2026-07-06 03:30:05] harvest start: 3 log(s), digest=   18255B
一回性レビュー＋抽出基準に非該当。よってファイルは作成しませんでした。
[2026-07-06 03:30:45] harvest done (exit 0, created=0)

[2026-07-07 03:30:05] harvest start: 3 log(s), digest=   35008B
[2026-07-07 03:32:28] CREATED: note-bokumolily-factual-review
[2026-07-07 03:32:28] harvest done (exit 0, created=1)

Enter fullscreen mode Exit fullscreen mode

Four out of five days are created=0. A miss isn’t an anomaly — it’s proof the extraction criteria are working correctly. Duplicates of existing skills, one-off personal circumstances, and idle chat all get rejected. Because the LLM makes the rejection call autonomously, it doesn’t churn out junk.

Note: A created=0 harvest consumes almost no cost (Max quota). It’s just enough to launch claude -p, read the prompt, and return “nothing applicable.” “Misses being cheap” is part of why I can run this nightly batch every single night.

Summary

• Watermark method: extract only the diff logs with find -newer, keyed off the mtime of .harvest-watermark. Even with 3,349 logs, it processes only 3 per night
• Double budget cap: --max-budget-usd 1.20 + perl alarm 600 puts a ceiling on both LLM spend and execution time
• Noise filter up front: adding grep -v 'system-reminder' as preprocessing compresses the prompt to a fixed 45KB
• Writing via staging: mktemp -d lets the shell mediate, avoiding write contention against ~/.claude
• A miss is normal: with tight extraction criteria, a run of created=0 days is healthy behavior

I wrote about how skills crossing the 100 mark raised the importance of the curate side in the previous piece. Next time I plan to write about how these two batches piling up skills and memory started to make Claude Code slow to launch — auditing and reducing the amount of context injection.

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*