latest changes

This commit is contained in:
Joren de Graaf
2026-10-20 14:17:46 +02:00
parent dd1ee39afa
commit 03d7afc6a1
28 changed files with 1929 additions and 15 deletions
+23
View File
@@ -0,0 +1,23 @@
# Card and progress-note attachments
Attachments are stored as private files with metadata in `CardAttachment`. A file belongs to one card, and optionally one `CardComment` progress note on that card. Moving a card keeps its files. Deleting a note or card cascades to its attachment records and removes stored files. Explicit deletion also removes the bytes. Files are snapshots; `source_path` and `source_commit` record provenance but do not cause automatic refresh.
## Configuration
Set `PROJECTS_PRIVATE_MEDIA_ROOT` to a persistent directory outside any web server static or public media alias. The default is `private_media/` under the repository for local development. Back up this directory alongside the database. The application account needs read/write access. No `MEDIA_URL` or public media route is configured.
Limits default to 10 MiB per file, 50 files and 100 MiB per card. Override with `PROJECTS_ATTACHMENT_MAX_BYTES`, `PROJECTS_ATTACHMENT_MAX_FILES` and `PROJECTS_ATTACHMENT_MAX_CARD_BYTES`. MCP base64 upload/fetch has a stricter 2 MiB limit to bound tool payloads. The accepted extensions are UTF-8 text/code (`txt`, `md`, `py`, `js`, `ts`, `tsx`, `jsx`, `json`, `css`, `html`, `xml`, `csv`, `yaml`, `yml`, `sh`, `diff`, `patch`) and PNG, JPEG, WebP, PDF. HTML is always served as plain text. SVG and archives are rejected. Files are not extracted or executed. Download responses use `Content-Disposition: attachment`, `nosniff`, and `no-store`.
## Web use
On a card page, upload multiple files in **Card files** or beneath a specific progress note. The form works without JavaScript; JavaScript adds drag and drop, progress and inline error feedback. The card and note sections share a simple list. Safe images up to 1 MiB have a thumbnail and preview link; UTF-8 text up to 1 MiB has a text preview. Other files download. Rename changes only the display title. Deleting requires confirmation.
All reads require project access. Upload, rename and delete require editor or owner access. Django session forms use CSRF. An attachment ID alone does not grant access.
## MCP transfer
`attachments_list(card_id, comment_id?, limit?, offset?)` returns metadata for card and note files. `attachments_get(attachment_id)` returns metadata. `attachments_fetch(attachment_id)` returns exact `content_base64`, SHA-256, and decoded `text` for UTF-8 files. `attachments_upload(card_id, filename, content_base64, comment_id?, title?, description?, source_path?, source_commit?)` transfers bytes to the server. `attachments_update` changes display metadata; `attachments_delete` removes bytes and metadata. Read tools require `projects:read`; writes require `projects:write`. Each tool checks card access. A remote server never reads a caller's filesystem path.
For a local repository snapshot, read the file locally, calculate its SHA-256, base64 encode the bytes, then call `attachments_upload` with the target card and progress-note IDs. Include a repository-relative `source_path` and a Git commit SHA when available. Compare the returned SHA-256; `attachments_fetch` lets another client verify exact bytes. If a working-tree file differs from the recorded commit, describe that in the progress note; `source_commit` alone cannot establish the working-tree state.
The tool output uses standard base64 in structured JSON so clients can reconstruct binary files. Client support for passing generated image bytes into a tool, exposing large tool payloads, and rendering fetched images must be checked against a deployed MCP connection. The local tests exercise the actual MCP tool call path for text bytes and permission denial; they cannot establish compatibility with a remote ChatGPT client before deployment.
+17
View File
@@ -0,0 +1,17 @@
# Companion movement prototype
`dashboard/static/dashboard/companion.js` is a browser-only, framework-independent movement core. A host creates `new BrowserCompanion({ surfaces, name, reducedMotion, config, debug, debugSpeed })`; `surfaces()` returns `{ element, edge: 'top' | 'bottom', card?, explicit? }` entries. The host owns DOM discovery, preferences, and lifecycle. The core owns geometry validation, the directed route graph, movement and the coloured-square renderer. A host may supply a renderer implementing `render(position, state, name)` and `destroy()`.
`bodySize` is the collision-body height in CSS pixels, independent of future sprite padding. Distance settings ending in `S` are multiples of that size. Speed settings ending in `SPerSec` use those units per second. Durations are milliseconds. `configure()` clamps invalid numeric overrides and ignores unknown keys. The defaults are initial visual tuning values, not animal measurements.
An ordinary nested ledge needs a visible computed top border of at least 1 CSS pixel. Climb edges need visible left or right borders. Outer widget top and bottom edges are explicit known surfaces. Interactive controls are excluded unless a host deliberately opts one in. Geometry reads are performed on host refresh events, not each animation frame. The Project Manager adapter refreshes after grid changes, scrolling, resizing and widget mount or removal.
Directed routes include short jumps, steps, walks, side climbs and bounded drops. A short attach hop can bridge the normal grid gap to a visible climb edge if its rise stays within `maxJumpRiseS`; this allows a safe drop to a lower card to be followed by a climb back. Downward climbing is not enabled. If a destination cannot be reached through a complete safe route, the companion idles or patrols its current ledge. Page scroll keeps its position attached to the source element. A disappearing destination cancels the move and recovers only to a nearby valid ledge; otherwise the box is hidden until a safe anchor appears. Background tabs pause motion, and reduced motion stops autonomous movement.
The activity scheduler has calm, active and resting modes. Normal calm idle lasts 12–20 seconds, sleep 20–45 seconds, and purring/batting 2.2–3.8 seconds. Active bursts plan 2–5 safe transitions before returning to calm; short movement pauses do not choose unrelated passive states. Emote and sleep cooldowns prevent rapid cycling. Rest becomes more likely after repeated bursts. These bounds and activity weights are host-overridable through `config`. For local debugging only, `?companionFast` scales dwell and cooldown time by 10 without changing movement safety or turning on the visual debug overlay.
Append `?companionDebug` to the dashboard URL to show ledges, climb edges, graph transitions, rejected element marks, active route and movement limits. The overlay is disabled by default and never receives pointer events.
The debug label also shows the current state reason, recent timestamped transitions, mode, remaining burst steps, next dwell timer, active motion phase, and the number of safe jump/drop edges available from the current platform. `landing` lasts 260 ms and is entered only after an airborne jump, drop, or climb hop. Walking, pull-ups, reflow, and completed routes settle to `idle` instead.
The dependency-free browser checks live in `dashboard/tests/companion_movement.html`. Serve the repository root with a local static server and open that page; every line should report `PASS`. The Django dashboard suite remains runnable with `.venv/bin/python manage.py test dashboard`.
@@ -0,0 +1,81 @@
# Project documentation and MCP evaluation
Research snapshot: 20 October 2026. Repository inspected at `dd1ee39` with uncommitted local changes; this report makes no runtime changes.
## Recommendation
**Go for a small native Django documentation module; do not install or fork a wiki for the MVP.** Treat the repository's Markdown files as the source of truth. Import a versioned, project-scoped snapshot into the database for browsing and MCP retrieval, then show its source commit and freshness on every read. This gives agents a bounded, searchable map of project context without rescanning the repository each time. The database index is a cache of a declared Git snapshot, not an independent authority.
The first milestone should be **read-only documentation**: upload a Markdown ZIP, review its preview, publish the imported snapshot, browse a tree, search, and use `docs_list`, `docs_search`, and `docs_read` over MCP. Do not accept MCP edits until there is a concrete export/reconciliation workflow. Later edits can be proposed revisions with an explicit base source commit and document hash, downloaded as Markdown ZIP or applied through a separately approved Git workflow. Never silently alter the published snapshot or repository.
## Evidence in this project
| Existing part | Reuse / gap |
| --- | --- |
| `projects/models.py`: `Project` has `git_repository_url`, owner, and `Membership`; `Card` and `CardRevision` store Markdown and revisions | Reuse project identity and access policy. The URL is metadata only; neither Django nor MCP clones repositories. Documents need their own path, source commit/hash, and version model, separate from Kanban cards. |
| `projects/services.py`: `accessible()` and `project_for(..., write=True)` | Reuse for **every** document query and mutation. Owner/editor can import; viewer can read. Do not rely only on template visibility or Django's global model permissions. |
| `projects/templatetags/project_markdown.py` | Python-Markdown plus `nh3` already sanitizes rendered HTML. Extract a shared renderer function and use it for docs; add a controlled link resolver for relative `.md` links. Current sanitizer allows only `http`, `https`, and `mailto` URLs, so ordinary relative links are stripped until resolved. |
| `projects/views.py`, `projects/urls.py` | Authenticated project pages and CSRF-protected POST patterns already exist. Add project-scoped docs routes and a navigation link without changing the board data model. |
| `mcp_server/server.py` | MCP uses `MCPServer`, `projects:read`/`projects:write`, `_project`, `_safe`, and bounded list patterns. Add document tools through the same service layer. Current `cards_search` and `search_global` return full Markdown per hit; docs search should return short snippets and metadata to stay useful at scale. |
| `config/settings.py` | SQLite is the configured database. Design search to work with SQLite first; consider SQLite FTS5 after profiling. PostgreSQL search is not a current dependency. |
| `projects/attachments.py`, `docs/card-attachments.md` | Useful reference for private storage, upload limits, and provenance, but card attachments do not provide a navigable documentation tree or searchable document text. |
## Candidate comparison
| Candidate | Current release / licence | Fit and integration cost | Verdict |
| --- | --- | --- | --- |
| [django-wakawaka](https://pypi.org/project/django-wakawaka/) | 1.7, 7 Feb 2026; MIT. Its published compatibility matrix ends at Django 5.2, with no declared Django 6.1 test. | Small dependency, but its [model](https://github.com/bartTC/django-wakawaka/blob/main/wakawaka/models.py) has a global `slug`, no project foreign key; [views](https://github.com/bartTC/django-wakawaka/blob/main/wakawaka/views.py) query all pages and use global Django permissions. Its [page template](https://github.com/bartTC/django-wakawaka/blob/main/wakawaka/templates/wakawaka/page.html) uses `urlize|wikify|linebreaks`, not Markdown. Replacing the template is easy, but adding project isolation, paths, imports, source snapshots, and MCP would replace much of its behavior. The inspected revision view also looks up a revision by ID without checking it belongs to the requested page, making a direct integration unsuitable for project-private docs. | No dependency or fork for MVP. Retest Django 6.1 only if considering it later. |
| [django-wiki (`wiki`)](https://pypi.org/project/wiki/) | 0.13.0, 31 Jul 2026; GPL-3.0. [Package metadata](https://github.com/django-wiki/django-wiki/blob/main/pyproject.toml) requires `Django>=4.2,<6.1` and several app dependencies. | Mature, revisioned wiki with Markdown and tree features, but dependency constraint conflicts with this repo's `Django>=6.1,<6.2`. Its plugin and renderer stack would add a second content system. | No-go on current stack. |
| [Waliki](https://pypi.org/project/waliki/) | 0.8.1, 26 Mar 2017; BSD. | Git-backed premise is attractive, but its published documentation targets Django 1.8/1.9-era Python. No evidence of current Django 6.1 compatibility. | No-go; major port/fork risk. |
| [djiki](https://pypi.org/project/djiki/) | 0.9, 1 May 2022; BSD-3-Clause. | Its inspected models use globally unique page titles/language, not project-scoped file paths. Published classifiers stop at Python 3.10, with no current Django 6.1 evidence. | No-go for this use case. |
| Native Django docs module | No new wiki dependency or third-party licence | Smallest domain fit: project/path uniqueness, Git provenance, import preview, permission checks, MCP output shape, and existing sanitizer can be designed together. | Recommended. |
The Wakawaka renderer verdict is especially clear: there is no Markdown renderer to replace. A template override could call `safe_markdown`, but the default CamelCase substitution and global data model still conflict with normal repository Markdown. Forking it would create long-term maintenance without removing the custom work.
## Proposed shape
1. `ProjectDocument`: `project`, normalized case-preserving POSIX `path` (e.g. `README.md`, `docs/architecture/mcp.md`), raw Markdown, content SHA-256, title extracted from heading or filename, `revision`, source commit, imported time, and status. Unique `(project, path)`; index `(project, path)` and search fields. A separate `DocumentRevision` stores immutable previous content, actor, timestamp, and source/proposal type if editing is added.
2. `DocumentSnapshot` or import batch records project, source commit supplied by uploader, ZIP hash, imported time, counts, and outcome. Record whether the commit was verified against a trusted repository checkout; an uploader-supplied SHA alone is **unverified provenance**. A published snapshot stays queryable until a complete replacement succeeds. Display freshness as "as of commit X / imported at Y"; do not claim current Git state unless verified.
3. Navigation is generated from paths, with `README.md` as landing page when present and `AGENTS.md` visible as a first-class context file. Preserve exact Markdown text for MCP. Resolve relative links against the source document's parent path; rewrite only links to imported documents and anchors. External links retain the current sanitizer's URL restrictions. Report broken internal links on import.
4. A project docs page offers tree, search, raw/view modes, source commit, stale/unverified badge, and import/export controls for editors. Missing docs state explains how to upload an archive. Use normal login, CSRF, and project authorization. Do not show a success state after a partial import failure.
### MCP contract and agent context
| Tool | Return shape / controls |
| --- | --- |
| `docs_list(project, prefix?, cursor?, limit?)` | Paths, titles, hashes, revision, snapshot/commit, modified time; stable path order, at most 100 items, cursor pagination. |
| `docs_search(query, project?, cursor?, limit?)` | Search only accessible projects; path/title, bounded excerpt, project ID, commit/hash, and direct read locator. Limit query and excerpt size. Search metadata first, full text second. Do not return whole files per hit. |
| `docs_read(project, path, revision?, start_line?, line_count?)` | Exact Markdown or a bounded line window, total line count, hash, source commit, verified/stale status, canonical path. Reject invalid paths and excessive output; permit explicit paging for large files. |
For the user's memory goal, start each work session with `docs_list` or targeted `docs_search`, then `docs_read` for the few relevant files. A small curated `docs/PROJECT_CONTEXT.md` or existing `AGENTS.md` can point to architecture, conventions, deployment, and current decisions; the index should not auto-concatenate all documents into every response. Keep path/hash/commit in replies so the agent can tell when context is old and inspect live source before changing code. The docs index reduces repeated discovery, but it cannot guarantee that unimported code changes are reflected.
Later `docs_create`/`docs_update` should require `projects:write`, project editor rights, normalized path, size cap, and `expected_revision` plus `base_source_commit`/content hash. Store proposals separately from published Git-backed content. Guard deletion as a proposal or tombstone, with explicit export and conflict handling. No direct Git fetch/push, credentials, or arbitrary filesystem paths in MVP.
## ZIP workflow and controls
Preview before commit: accept only UTF-8 `.md` files from declared roots (`README.md`, `AGENTS.md`, `docs/**` or a selected docs subtree); display additions, changes, removals, broken links, and conflict warnings. Reject duplicate normalized paths, absolute paths, `..`, backslashes or ambiguous separators, symlinks and other nonregular entries, hidden control files, and oversized archives. Bound compressed bytes, expanded bytes, file count, per-file bytes, compression ratio, and nesting depth; inspect ZIP metadata before reading and stream under limits. Normalize Unicode and define case-collision behavior. Require the expected previous snapshot ID/commit before a re-import; publish atomically. Export Markdown paths plus a manifest with source commit and hashes; exclude server-only metadata and secrets. Import must never extract directly into the repository or execute contents.
## Work estimate and stages
These are rough engineering estimates, not commitments; integration and review may change them.
| Stage | Scope | Estimate |
| --- | --- | --- |
| 1 | Models/migrations, service authorization, path validation, snapshot provenance, tests | 2–3 days |
| 2 | ZIP preview/import/export, limits, link checks, atomic replacement, tests | 2–4 days |
| 3 | Browsing/search UI, renderer/link adapter, accessible states | 2–3 days |
| 4 | Read-only MCP tools, pagination/output limits, permission tests | 1–2 days |
| Later | Proposed edits/reconciliation/export, conflict handling | 2–4 days after workflow decision |
An MVP is roughly **7–12 engineering days**. The main uncertainties are archive scale, exact desired Git provenance, and whether a read-only import meets the workflow. No implementation is blocked by a missing wiki library.
## Decisions before implementation
- Choose the ingestion boundary: manual ZIP upload first (recommended) or trusted server-side checkout. The current Git URL is metadata and must not become an arbitrary fetch target without host allowlisting, credential design, and SSRF controls.
- Choose whether the initial import mirrors only tracked documentation files or allows uploaded notes not in Git. If Git is authoritative, label non-Git notes separately.
- Decide when a snapshot is considered stale. An import timestamp alone cannot prove repository freshness; a trusted commit check or explicit manual refresh is needed.
- Decide whether cross-project `docs_search` should be enabled for every authorized membership or opt-in for sensitive projects. The existing MCP `search_global` searches across all accessible projects, so the former is consistent with present behavior.
## Sources and inspection method
Local sources: `requirements.txt`, `config/settings.py`, `projects/models.py`, `projects/services.py`, `projects/views.py`, `projects/templatetags/project_markdown.py`, `mcp_server/server.py`, `projects/attachments.py`, and `README.md`. Upstream release metadata came from the linked PyPI pages; package source distributions were inspected directly for models, views, templates, and dependency declarations. These findings establish design fit, not runtime compatibility tests against Django 6.1.