coopcloud_code/home
0
0
Fork 0
Code Issues 2 Pull requests Projects Releases Packages Wiki Activity Actions

Scaling architecture: marketplace + hero_ledger + hero_compute + MOS for millions of farmers #72

New issue
Open
opened 2026-04-10 20:42:11 +00:00 by mik-tf · 8 comments
mik-tf commented 2026-04-10 20:42:11 +00:00
Member
Copy link

Summary

Design and execution plan for scaling the Mycelium compute marketplace from ~10 nodes (today) to millions of farmers and users on devnet, keeping mainnet launch one infra rollout away.

Tracking doc

Full spec: https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_deploy/src/branch/development_mik02/docs/scaling_architecture.md

The doc is the single source of truth for this initiative. Everything below is a summary — read the doc for detail, acceptance criteria, file paths, and reasoning.

Vision

  • Farmer plugs USB into any x86 box, powers on, node auto-publishes in ~45 seconds of human effort.
  • User rents a slice, VM boots on real farmer hardware with cryptographically verified identity at every hop.
  • Marketplace backend is stateless — fully rehydratable from hero_ledger alone.
  • hero_ledger is the only source of truth. Marketplace, explorer, and node are caches/orchestrators.
  • Horizontal scale: more backend instances, more regional explorers, more nodes — no central chokepoint.

The 4 gaps

  1. hero_ledger gateway client missing hosting contract methods
  2. marketplace contract accepts any node_id without cross-checking hosting
  3. hero_compute_server has no crypto identity (hostname-based, unsigned heartbeats)
  4. MOS has no per-farmer config injection

Phase checklist

  • Phase 0 — Spec + branches + this issue
  • Phase 1 — Real-hardware MVP (crude bridge + QEMU/Docker simulated node) — closes #71
  • Phase 2 — hero_ledger gateway hosting client methods (upstream PR)
  • Phase 3.0 — HARD CHECKPOINT: NEAR implicit accounts on devnet (go/no-go)
  • Phase 3 — Node crypto identity in hero_compute (upstream PR)
  • Phase 4 — mkt-prepare-usb tool
  • Phase 5 — Auto-pairing endpoint + pending nodes UI
  • Phase 6 — Pricing templates (marketplace-side)
  • Phase 7 — FarmNode → read-through cache (big simplification)
  • Phase 8 — Regional explorers (design doc only, implementation deferred)

Cross-repo branch: development_mik02

Active on all 6 repos:

Repo Purpose Maintainer
https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_backend Axum backend mik-tf
https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_frontend Dioxus SPA mik-tf
https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_deploy Docker, tests, docs mik-tf
https://forge.ourworld.tf/lhumina_code/hero_ledger NEAR-compatible blockchain lhumina team (PR)
https://forge.ourworld.tf/lhumina_code/hero_compute VM provisioning + explorer lhumina team (PR)
https://forge.ourworld.tf/geomind_code/mos_builder MOS image builder geomind team (PR)

Ground rules

  • Never push to development on any repo without PR review
  • Upstream PRs (hero_ledger, hero_compute, mos_builder): we open, maintainers review and merge
  • Marketplace repos: mik-tf merges after own review
  • No Cargo.toml path deps — local dev uses ~/.cargo/config.toml paths = [...] override
  • 7-layer test pyramid green before every PR

Relates to

Status updates

This issue will be updated as each phase completes. See the tracking doc for the live source of truth on architecture decisions and file-level changes.

## Summary Design and execution plan for scaling the Mycelium compute marketplace from ~10 nodes (today) to millions of farmers and users on devnet, keeping mainnet launch one infra rollout away. ## Tracking doc **Full spec**: https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_deploy/src/branch/development_mik02/docs/scaling_architecture.md The doc is the single source of truth for this initiative. Everything below is a summary — read the doc for detail, acceptance criteria, file paths, and reasoning. ## Vision - Farmer plugs USB into any x86 box, powers on, node auto-publishes in ~45 seconds of human effort. - User rents a slice, VM boots on real farmer hardware with cryptographically verified identity at every hop. - Marketplace backend is stateless — fully rehydratable from `hero_ledger` alone. - `hero_ledger` is the only source of truth. Marketplace, explorer, and node are caches/orchestrators. - Horizontal scale: more backend instances, more regional explorers, more nodes — no central chokepoint. ## The 4 gaps 1. `hero_ledger` gateway client missing `hosting` contract methods 2. `marketplace` contract accepts any `node_id` without cross-checking `hosting` 3. `hero_compute_server` has no crypto identity (hostname-based, unsigned heartbeats) 4. MOS has no per-farmer config injection ## Phase checklist - [x] Phase 0 — Spec + branches + this issue - [ ] Phase 1 — Real-hardware MVP (crude bridge + QEMU/Docker simulated node) — **closes #71** - [ ] Phase 2 — `hero_ledger` gateway hosting client methods (upstream PR) - [ ] Phase 3.0 — **HARD CHECKPOINT**: NEAR implicit accounts on devnet (go/no-go) - [ ] Phase 3 — Node crypto identity in `hero_compute` (upstream PR) - [ ] Phase 4 — `mkt-prepare-usb` tool - [ ] Phase 5 — Auto-pairing endpoint + pending nodes UI - [ ] Phase 6 — Pricing templates (marketplace-side) - [ ] Phase 7 — `FarmNode` → read-through cache (big simplification) - [ ] Phase 8 — Regional explorers (design doc only, implementation deferred) ## Cross-repo branch: `development_mik02` Active on all 6 repos: | Repo | Purpose | Maintainer | |------|---------|-----------| | https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_backend | Axum backend | mik-tf | | https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_frontend | Dioxus SPA | mik-tf | | https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_deploy | Docker, tests, docs | mik-tf | | https://forge.ourworld.tf/lhumina_code/hero_ledger | NEAR-compatible blockchain | lhumina team (PR) | | https://forge.ourworld.tf/lhumina_code/hero_compute | VM provisioning + explorer | lhumina team (PR) | | https://forge.ourworld.tf/geomind_code/mos_builder | MOS image builder | geomind team (PR) | ## Ground rules - **Never push to `development`** on any repo without PR review - **Upstream PRs** (`hero_ledger`, `hero_compute`, `mos_builder`): we open, maintainers review and merge - **Marketplace repos**: mik-tf merges after own review - **No `Cargo.toml` path deps** — local dev uses `~/.cargo/config.toml` `paths = [...]` override - **7-layer test pyramid green** before every PR ## Relates to - Closes https://forge.ourworld.tf/mycelium_code/home/issues/71 via Phase 1 - Feeds https://forge.ourworld.tf/mycelium_code/home/issues/55 (P8-P13 infra) via Phase 7 - Umbrella: https://forge.ourworld.tf/mycelium_code/home/issues/40 ## Status updates This issue will be updated as each phase completes. See the tracking doc for the live source of truth on architecture decisions and file-level changes.
mik-tf commented 2026-04-10 20:59:54 +00:00
Author
Member
Copy link

Session 1 progress report — 2026-04-10

Done

  • Phase 0 complete: spec doc (793 lines, 9 sections, 8 phases), this tracking issue, development_mik02 branches on all 6 repos, ~/.cargo/config.toml path overrides (no Cargo.toml pollution)
  • Phase 1 code complete: backend + frontend committed on development_mik02. Crude bridge — farmer pastes mycelium IPv6 into the Register Node form, backend queries hero_compute_explorer.node_list(), filters client-side, auto-fills capacity from real heartbeat data, creates FarmNode + best-effort hero_ledger listing. 4 files changed on backend (+260 LOC), 1 file on frontend (+53 LOC, -14 LOC). cargo check clean, 25 backend unit tests pass
  • Phase 3.0 HARD CHECKPOINT ✅ PASSED: verified by existing evidence — the marketplace devnet account is itself an implicit account (a63dc8c8... is exactly hex(pubkey)). Created 2026-04-01, funded with 10k SPORE, actively used for SPORE transfers. Proves end-to-end that implicit accounts work on devnet. Phase 3 can proceed as designed.
  • Phase 2 scope recalibrated: original plan (add hosting.rs to gateway client + server) is ~600 LOC upstream. Deferred to Phase 2.5 as a proper upstream follow-up. hero_compute Phase 3 uses direct near-jsonrpc-client for its writes, no new gateway dep. Spec doc updated.

Deferred to next session

  • Phase 1 deploy + validate on dev VM: code is committed, but deploying from development_mik02 requires docker-compose.override.yml or temporary :development tag overwrite. Deferred to a dedicated deploy session to avoid risking the stable dev environment mid-development.
  • Phase 3, 4, 5: not started. Phase 3.0 unblocked Phase 3; Phase 2.5 follow-up deferred; Phase 4 (mkt-prepare-usb) is a standalone tool independent of other phases.
  • MOS build: blocked on sudo apt install -y upx-ucl (host dep). Not on Phase 0-3 critical path — Phase 1 validation can use the existing dev VM hero_compute instead of a real MOS boot.

Key links

Next session starting points

  1. Deploy Phase 1 to dev VM (separate tag or docker-compose override), curl the new endpoint, run 398-test pyramid, open backend + frontend PRs
  2. Phase 3: node crypto identity in hero_compute_server — ed25519 keypair gen + sig verification + first-boot hook calling hosting.farm_create + hosting.node_register via near-jsonrpc-client
  3. Phase 4: mkt-prepare-usb standalone tool
  4. Phase 5: auto-pairing endpoint (signed payload validation, pending FarmNode state, Publish action)
  5. Phase 2.5 upstream: open hero_ledger PR adding hosting.rs client + server methods (separate workstream)

Ground rules being enforced

  • Zero pushes to development on any repo — only development_mik02
  • No Cargo.toml path deps — all local overrides via ~/.cargo/config.toml
  • No direct merges to upstream (hero_ledger, hero_compute, mos_builder) — PRs only, maintainers review
  • Every phase green on the 7-layer test pyramid before PR opens
## Session 1 progress report — 2026-04-10 ### Done - **Phase 0** complete: spec doc (793 lines, 9 sections, 8 phases), this tracking issue, `development_mik02` branches on all 6 repos, `~/.cargo/config.toml` path overrides (no Cargo.toml pollution) - **Phase 1 code** complete: backend + frontend committed on `development_mik02`. Crude bridge — farmer pastes mycelium IPv6 into the Register Node form, backend queries `hero_compute_explorer.node_list()`, filters client-side, auto-fills capacity from real heartbeat data, creates FarmNode + best-effort hero_ledger listing. 4 files changed on backend (+260 LOC), 1 file on frontend (+53 LOC, -14 LOC). `cargo check` clean, 25 backend unit tests pass - **Phase 3.0 HARD CHECKPOINT** ✅ PASSED: verified by existing evidence — the marketplace devnet account is itself an implicit account (`a63dc8c8...` is exactly `hex(pubkey)`). Created 2026-04-01, funded with 10k SPORE, actively used for SPORE transfers. Proves end-to-end that implicit accounts work on devnet. Phase 3 can proceed as designed. - **Phase 2 scope recalibrated**: original plan (add `hosting.rs` to gateway client + server) is ~600 LOC upstream. Deferred to Phase 2.5 as a proper upstream follow-up. hero_compute Phase 3 uses direct `near-jsonrpc-client` for its writes, no new gateway dep. Spec doc updated. ### Deferred to next session - **Phase 1 deploy + validate on dev VM**: code is committed, but deploying from `development_mik02` requires docker-compose.override.yml or temporary `:development` tag overwrite. Deferred to a dedicated deploy session to avoid risking the stable dev environment mid-development. - **Phase 3, 4, 5**: not started. Phase 3.0 unblocked Phase 3; Phase 2.5 follow-up deferred; Phase 4 (mkt-prepare-usb) is a standalone tool independent of other phases. - **MOS build**: blocked on `sudo apt install -y upx-ucl` (host dep). Not on Phase 0-3 critical path — Phase 1 validation can use the existing dev VM hero_compute instead of a real MOS boot. ### Key links - Spec doc (updated with Phase 2 recalibration + Phase 3.0 pass): https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_deploy/src/branch/development_mik02/docs/scaling_architecture.md - Backend Phase 1 commit: https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_backend/src/branch/development_mik02 - Frontend Phase 1 commit: https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_frontend/src/branch/development_mik02 - Deploy repo `development_mik02`: https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_deploy/src/branch/development_mik02 ### Next session starting points 1. Deploy Phase 1 to dev VM (separate tag or docker-compose override), curl the new endpoint, run 398-test pyramid, open backend + frontend PRs 2. Phase 3: node crypto identity in `hero_compute_server` — ed25519 keypair gen + sig verification + first-boot hook calling `hosting.farm_create` + `hosting.node_register` via `near-jsonrpc-client` 3. Phase 4: `mkt-prepare-usb` standalone tool 4. Phase 5: auto-pairing endpoint (signed payload validation, pending FarmNode state, Publish action) 5. Phase 2.5 upstream: open hero_ledger PR adding `hosting.rs` client + server methods (separate workstream) ### Ground rules being enforced - Zero pushes to `development` on any repo — only `development_mik02` - No `Cargo.toml` path deps — all local overrides via `~/.cargo/config.toml` - No direct merges to upstream (`hero_ledger`, `hero_compute`, `mos_builder`) — PRs only, maintainers review - Every phase green on the 7-layer test pyramid before PR opens
mik-tf commented 2026-04-10 23:28:29 +00:00
Author
Member
Copy link

Phase 1 complete (2026-04-10)

Deployed :development_mik02 images for backend and frontend to dev-app.projectmycelium.org via docker-compose override on the dev VM. Real smoke test against the on-host hero_compute node succeeded end-to-end — FarmNode created from explorer heartbeat with compute_node_sid linkage in grid_data for Phase 5 pairing.

PRs:

Test results on dev-app: 444 passed / 7 pre-existing failures unrelated to Phase 1 (DEMO_KYC auto-verify + local-only unix-socket MCP tests). Full breakdown in mycelium_code/home#71 (now closed).

Phase checklist update:

  • Phase 0 — Spec + branches + issue
  • Phase 1 — Real-hardware MVP (crude bridge + simulated node) — closes #71
  • Phase 2 — hero_ledger gateway hosting client methods (upstream PR) — deferred to Phase 2.5 parallel workstream
  • Phase 3.0 — NEAR implicit accounts checkpoint
  • Phase 3 — Node crypto identity in hero_compute (next)
  • Phase 4 — mkt-prepare-usb tool
  • Phase 5 — Auto-pairing endpoint + pending nodes UI
  • Phase 6 — Pricing templates
  • Phase 7 — FarmNode → read-through cache
  • Phase 8 — Regional explorers (design doc only)

Next up: Phase 3 — node crypto identity in hero_compute_server (upstream PR).

## Phase 1 complete (2026-04-10) Deployed `:development_mik02` images for backend and frontend to `dev-app.projectmycelium.org` via docker-compose override on the dev VM. Real smoke test against the on-host hero_compute node succeeded end-to-end — FarmNode created from explorer heartbeat with `compute_node_sid` linkage in `grid_data` for Phase 5 pairing. **PRs**: - Backend: https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_backend/pulls/1 - Frontend: https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_frontend/pulls/1 **Test results on dev-app**: 444 passed / 7 pre-existing failures unrelated to Phase 1 (DEMO_KYC auto-verify + local-only unix-socket MCP tests). Full breakdown in https://forge.ourworld.tf/mycelium_code/home/issues/71 (now closed). **Phase checklist update**: - [x] Phase 0 — Spec + branches + issue - [x] **Phase 1 — Real-hardware MVP (crude bridge + simulated node)** — closes #71 - [ ] Phase 2 — `hero_ledger` gateway hosting client methods (upstream PR) — deferred to Phase 2.5 parallel workstream - [x] Phase 3.0 — NEAR implicit accounts checkpoint - [ ] Phase 3 — Node crypto identity in `hero_compute` (next) - [ ] Phase 4 — `mkt-prepare-usb` tool - [ ] Phase 5 — Auto-pairing endpoint + pending nodes UI - [ ] Phase 6 — Pricing templates - [ ] Phase 7 — FarmNode → read-through cache - [ ] Phase 8 — Regional explorers (design doc only) Next up: Phase 3 — node crypto identity in `hero_compute_server` (upstream PR).
mik-tf commented 2026-04-11 00:46:42 +00:00
Author
Member
Copy link

Dev VM heartbeat fix + upstream PR (side-quest from Phase 1)

While validating Phase 1, we found the dev VM hero_compute node was flagged offline even though the server was running. Root cause was a combination of:

  1. hero_compute_server was started without MYCELIUM_IP env var set — so its heartbeats sent an empty string for the field.
  2. ExplorerService.node_heartbeat in hero_compute_explorer unconditionally overwrote node.mycelium_ip with the incoming value — so every heartbeat nuked the correct IP that was there from the initial node registration.

End effect: node kept showing online briefly after restart, then flipped to offline forever (or showed online with empty mycelium_ip, which breaks the marketplace ComputeClient::lookup_by_mycelium_ip filter used by the Phase 1 add-node-from-explorer flow).

Dev VM fix (runtime, already applied)

  • Replaced the broken hero-compute.service systemd unit (which called hero_compute --start --mode local — broken with an opaque Symbolic link loop (os error 40)) with three new clean units: hero-compute-explorer, hero-compute-server, hero-compute-ui. Each runs a single binary with Restart=on-failure.
  • The hero-compute-server unit sets EXPLORER_ADDRESSES=/root/hero/var/sockets/hero_compute_explorer.sock + MYCELIUM_IP=46a:52b7:d2c2:4416:ff0f:5892:d922:50dc via Environment=.
  • Old hero-compute.service disabled and kept on disk as .bak-phase1 for rollback.
  • Killed an orphan bash wrapper + duplicate server from a previous manual restart that were both holding LISTEN fds on the same socket.

End-to-end verified: marketplace Phase 1 endpoint now returns the node as status: online with running_vms correctly populated.

Upstream fix (defensive, non-blocking)

lhumina_code/hero_compute#91 — branch development_mik02_heartbeat_preserve_ip:

  • hero_compute_explorer rpc.rs: only overwrite node.mycelium_ip when incoming is non-empty
  • hero_compute_server main.rs: WARN at startup if heartbeat sender is configured but MYCELIUM_IP env var is unset
  • New inline test heartbeat_preserve_tests::node_heartbeat_preserves_mycelium_ip_when_incoming_is_empty in rpc.rs (not tests.rs — that file is regenerated by build.rs every compile). Verified red→green.

CI green (Test / test (push) 1m25s, Test / test (pull_request) 1m30s). Waiting on maintainer review.

Non-blocking rationale

  • Marketplace backend has no cargo dependency on hero_compute — only TCP JSON-RPC to port 9002. So this upstream fix doesn't need to land for marketplace to benefit.
  • Dev VM is already working via the runtime env-var workaround in the new systemd units.
  • The code fix is purely defensive — protects against future cases where someone forgets to set MYCELIUM_IP.

Phase 3 (node crypto identity) can proceed independently on the same branch when that work starts.

## Dev VM heartbeat fix + upstream PR (side-quest from Phase 1) While validating Phase 1, we found the dev VM hero_compute node was flagged offline even though the server was running. Root cause was a combination of: 1. `hero_compute_server` was started without `MYCELIUM_IP` env var set — so its heartbeats sent an empty string for the field. 2. `ExplorerService.node_heartbeat` in `hero_compute_explorer` unconditionally overwrote `node.mycelium_ip` with the incoming value — so every heartbeat nuked the correct IP that was there from the initial node registration. End effect: node kept showing online briefly after restart, then flipped to offline forever (or showed online with empty `mycelium_ip`, which breaks the marketplace `ComputeClient::lookup_by_mycelium_ip` filter used by the Phase 1 add-node-from-explorer flow). ### Dev VM fix (runtime, already applied) - Replaced the broken `hero-compute.service` systemd unit (which called `hero_compute --start --mode local` — broken with an opaque `Symbolic link loop (os error 40)`) with three new clean units: `hero-compute-explorer`, `hero-compute-server`, `hero-compute-ui`. Each runs a single binary with `Restart=on-failure`. - The `hero-compute-server` unit sets `EXPLORER_ADDRESSES=/root/hero/var/sockets/hero_compute_explorer.sock` + `MYCELIUM_IP=46a:52b7:d2c2:4416:ff0f:5892:d922:50dc` via `Environment=`. - Old `hero-compute.service` disabled and kept on disk as `.bak-phase1` for rollback. - Killed an orphan bash wrapper + duplicate server from a previous manual restart that were both holding LISTEN fds on the same socket. End-to-end verified: marketplace Phase 1 endpoint now returns the node as `status: online` with running_vms correctly populated. ### Upstream fix (defensive, non-blocking) https://forge.ourworld.tf/lhumina_code/hero_compute/pulls/91 — branch `development_mik02_heartbeat_preserve_ip`: - `hero_compute_explorer` `rpc.rs`: only overwrite `node.mycelium_ip` when incoming is non-empty - `hero_compute_server` `main.rs`: WARN at startup if heartbeat sender is configured but `MYCELIUM_IP` env var is unset - New inline test `heartbeat_preserve_tests::node_heartbeat_preserves_mycelium_ip_when_incoming_is_empty` in rpc.rs (not tests.rs — that file is regenerated by build.rs every compile). Verified red→green. CI green (Test / test (push) 1m25s, Test / test (pull_request) 1m30s). Waiting on maintainer review. ### Non-blocking rationale - Marketplace backend has no cargo dependency on hero_compute — only TCP JSON-RPC to port 9002. So this upstream fix doesn't need to land for marketplace to benefit. - Dev VM is already working via the runtime env-var workaround in the new systemd units. - The code fix is purely defensive — protects against future cases where someone forgets to set `MYCELIUM_IP`. Phase 3 (node crypto identity) can proceed independently on the same branch when that work starts.
mik-tf commented 2026-04-11 01:33:54 +00:00
Author
Member
Copy link

Phase 3 prep: hosting contract deployment gap discovered and tracked

Before writing Phase 3 code, verified the Phase 3.0 HARD CHECKPOINT more thoroughly. The spec only verified that implicit accounts work on devnet (hex(pubkey) → SPORE transfers). It did NOT verify that the hosting contract itself is deployed. It is not.

Authenticated account.exists queries against https://ledger.dev.projectmycelium.com/rpc:

Account Exists
dev.hero, marketplace.dev.hero, spore.dev.hero, gld.dev.hero, usdh.dev.hero, faucet.dev.hero, identity.dev.hero, dns.dev.hero ✅
hosting.dev.hero ❌

Also checked alt names (hosting.hero, grid.dev.hero, tfgrid.dev.hero, farm.dev.hero, farms.dev.hero, compute.dev.hero, nodes.dev.hero, hosting.marketplace.dev.hero, hosting.mycelium, hosting) — all absent. Confirmed via rpc.discover on the gateway: 43 methods total, zero hosting methods.

Root cause: the hosting contract source exists at hero_ledger/contracts/hosting/ with tests, but the deploy script is at examples/rhai/contracts/deploy/11_deploy_hosting.rhai, NOT in scripts/rhai/provision/deploy_services.rhai. So it never ran as part of devnet provisioning.

Filed upstream

  • Issue: lhumina_code/hero_ledger#44
  • PR: lhumina_code/hero_ledger#45 — 13-line diff adding hosting to deploy_services.rhai, modeled on the existing DNS and Marketplace deploy blocks. Hosting contract verified to compile cleanly on wasm32-unknown-unknown from current source.
  • CI on PR #45 is failing with a runner-side git-over-https auth bug (same symptom on development commit bb54ee00 runs #189/#190). Not caused by the diff. Flagged for @scott.

Why this is NOT blocking Phase 3 implementation

  • We plan to implement and test Phase 3 using near-workspaces-rs — a NEAR sandbox that compiles and deploys contract WASM in-process. That is how NEAR contracts are idiomatically tested anyway. Strongermore deterministic tests than live-devnet calls.
  • Only the FINAL E2E validation step (real hero_compute_server on dev VM calling a real devnet hosting contract) actually needs the devnet deployment.
  • When @scott merges PR #45 and re-runs devnet provision, flipping from sandbox to devnet is a one-config-line change on our side.

Plan update

  1. ✅ PR #45 on hero_ledger opened (awaiting scott merge + devnet redeploy)
  2. ⏳ Phase 3 implementation on hero_compute development_mik02_phase3_crypto_identity branch with near-workspaces sandbox tests — starting now
  3. Phase 4 + Phase 5 marketplace-side work slots in as soon as Phase 3 PR is up

No blockers for continuing.

## Phase 3 prep: hosting contract deployment gap discovered and tracked Before writing Phase 3 code, verified the Phase 3.0 HARD CHECKPOINT more thoroughly. The spec only verified that **implicit accounts** work on devnet (hex(pubkey) → SPORE transfers). It did NOT verify that the **hosting contract itself** is deployed. It is not. Authenticated `account.exists` queries against `https://ledger.dev.projectmycelium.com/rpc`: | Account | Exists | |---|---| | `dev.hero`, `marketplace.dev.hero`, `spore.dev.hero`, `gld.dev.hero`, `usdh.dev.hero`, `faucet.dev.hero`, `identity.dev.hero`, `dns.dev.hero` | ✅ | | **`hosting.dev.hero`** | **❌** | Also checked alt names (hosting.hero, grid.dev.hero, tfgrid.dev.hero, farm.dev.hero, farms.dev.hero, compute.dev.hero, nodes.dev.hero, hosting.marketplace.dev.hero, hosting.mycelium, hosting) — all absent. Confirmed via `rpc.discover` on the gateway: 43 methods total, zero hosting methods. Root cause: the hosting contract source exists at `hero_ledger/contracts/hosting/` with tests, but the deploy script is at `examples/rhai/contracts/deploy/11_deploy_hosting.rhai`, NOT in `scripts/rhai/provision/deploy_services.rhai`. So it never ran as part of devnet provisioning. ### Filed upstream - Issue: https://forge.ourworld.tf/lhumina_code/hero_ledger/issues/44 - PR: https://forge.ourworld.tf/lhumina_code/hero_ledger/pulls/45 — 13-line diff adding hosting to `deploy_services.rhai`, modeled on the existing DNS and Marketplace deploy blocks. Hosting contract verified to compile cleanly on `wasm32-unknown-unknown` from current source. - CI on PR #45 is failing with a runner-side git-over-https auth bug (same symptom on `development` commit bb54ee00 runs #189/#190). Not caused by the diff. Flagged for @scott. ### Why this is NOT blocking Phase 3 implementation - We plan to implement and test Phase 3 using `near-workspaces-rs` — a NEAR sandbox that compiles and deploys contract WASM in-process. That is how NEAR contracts are idiomatically tested anyway. Strongermore deterministic tests than live-devnet calls. - Only the FINAL E2E validation step (real hero_compute_server on dev VM calling a real devnet hosting contract) actually needs the devnet deployment. - When @scott merges PR #45 and re-runs devnet provision, flipping from sandbox to devnet is a one-config-line change on our side. ### Plan update 1. ✅ PR #45 on hero_ledger opened (awaiting scott merge + devnet redeploy) 2. ⏳ Phase 3 implementation on `hero_compute` `development_mik02_phase3_crypto_identity` branch with near-workspaces sandbox tests — starting now 3. Phase 4 + Phase 5 marketplace-side work slots in as soon as Phase 3 PR is up No blockers for continuing.
mik-tf commented 2026-04-11 02:09:47 +00:00
Author
Member
Copy link

Phase 3 upstream PR ready for review (Steps 1-4)

lhumina_code/hero_compute#91 — now covers both the original heartbeat mycelium_ip preserve fix AND Phase 3 Steps 1-4 of the scaling architecture initiative.

What ships in #91

Step Commit Scope
Defensive fix ef17fe8 Preserve mycelium_ip on empty-string heartbeat
1 bd04f49 identity.rs — persistent ed25519 keypair + NEAR implicit account = hex(pubkey)
2 47b4f33 Startup integration — load identity on boot, log pubkey in ed25519:<base58> NEAR format
3 a92c6ce Signed heartbeats — canonical sorted-key JSON payload, sha256+ed25519 signing
4 38b1c8e Explorer signature verification + ALLOW_LEGACY_HEARTBEATS grace period + identity rotation guard
— fcf82c1, 54eb0ac cargo fmt + clippy fixes

Tests

44 unit tests green, fully covering:

  • Identity: keygen/persist/chmod/roundtrip/cross-key/tamper rejection (13)
  • Canonical payload: key sorting, byte stability, tamper detection (5)
  • Explorer verification: valid signatures accepted, tampered capacity/hostname rejected, cross-key rejection, timestamp skew (past + future), bad base64, non-hex account, wrong-length sig (9)
  • Existing cloud CRUD and config tests still pass (17)

CI

All green on 54eb0ac (Test / test push + pull_request both Successful in ~1m30s). Check, Test, Format, Clippy all pass. Awaiting @mahmoud review.

What does NOT ship in #91 (Step 5 deferred)

The first-boot registration hook that calls hosting.farm_create + hosting.node_register via direct near-jsonrpc-client is a separate follow-up PR on hero_compute. Rationale:

  1. It drags in the full near-jsonrpc-client / near-primitives / near-crypto dependency tree, which is heavy (~150+ transitive deps) and entirely orthogonal to identity+heartbeats. Mixing them makes PR #91 harder to review.

  2. The hosting contract is not deployed on devnet (hero_ledger#44 / PR #45 open). There is literally nothing to call against until that lands, so E2E validation cannot happen anyway. A sandbox-only near-workspaces-rs test is possible but adds another heavy test-dep tree to land code that will immediately be re-verified once devnet is updated.

  3. The Phase 3.5 first-boot hook cleanly stacks on top of the current PR once it merges. The public API on NodeIdentity (public_key_bytes, sign_raw, verify_raw, verify_hashed) already exists with #[allow(dead_code)] markers and doc comments explicitly pointing at Phase 3.5 — so the follow-up PR is purely additive.

Next actions in parallel (no blocker)

Moving to marketplace-side Phase 5 work on the development_mik02 line (pair endpoint + pending-nodes UI). This is pure marketplace code, direct-push, and can progress without waiting for any upstream merge. The hosting.is_farm_owner / node_get_by_identity verification inside the pair endpoint ships behind a feature flag (HOSTING_VERIFICATION=mock) that flips to real once devnet has the contract.

## Phase 3 upstream PR ready for review (Steps 1-4) https://forge.ourworld.tf/lhumina_code/hero_compute/pulls/91 — now covers both the original heartbeat mycelium_ip preserve fix AND Phase 3 Steps 1-4 of the scaling architecture initiative. ### What ships in #91 | Step | Commit | Scope | |---|---|---| | Defensive fix | `ef17fe8` | Preserve mycelium_ip on empty-string heartbeat | | 1 | `bd04f49` | `identity.rs` — persistent ed25519 keypair + NEAR implicit account = `hex(pubkey)` | | 2 | `47b4f33` | Startup integration — load identity on boot, log pubkey in `ed25519:<base58>` NEAR format | | 3 | `a92c6ce` | Signed heartbeats — canonical sorted-key JSON payload, sha256+ed25519 signing | | 4 | `38b1c8e` | Explorer signature verification + `ALLOW_LEGACY_HEARTBEATS` grace period + identity rotation guard | | — | `fcf82c1`, `54eb0ac` | cargo fmt + clippy fixes | ### Tests 44 unit tests green, fully covering: - Identity: keygen/persist/chmod/roundtrip/cross-key/tamper rejection (13) - Canonical payload: key sorting, byte stability, tamper detection (5) - Explorer verification: valid signatures accepted, tampered capacity/hostname rejected, cross-key rejection, timestamp skew (past + future), bad base64, non-hex account, wrong-length sig (9) - Existing cloud CRUD and config tests still pass (17) ### CI All green on 54eb0ac (Test / test push + pull_request both Successful in ~1m30s). Check, Test, Format, Clippy all pass. Awaiting @mahmoud review. ### What does NOT ship in #91 (Step 5 deferred) The first-boot registration hook that calls `hosting.farm_create` + `hosting.node_register` via direct `near-jsonrpc-client` is a **separate follow-up PR** on hero_compute. Rationale: 1. It drags in the full `near-jsonrpc-client` / `near-primitives` / `near-crypto` dependency tree, which is heavy (~150+ transitive deps) and entirely orthogonal to identity+heartbeats. Mixing them makes PR #91 harder to review. 2. **The hosting contract is not deployed on devnet** ([hero_ledger#44](https://forge.ourworld.tf/lhumina_code/hero_ledger/issues/44) / PR #45 open). There is literally nothing to call against until that lands, so E2E validation cannot happen anyway. A sandbox-only `near-workspaces-rs` test is possible but adds another heavy test-dep tree to land code that will immediately be re-verified once devnet is updated. 3. The Phase 3.5 first-boot hook cleanly stacks on top of the current PR once it merges. The public API on `NodeIdentity` (`public_key_bytes`, `sign_raw`, `verify_raw`, `verify_hashed`) already exists with `#[allow(dead_code)]` markers and doc comments explicitly pointing at Phase 3.5 — so the follow-up PR is purely additive. ### Next actions in parallel (no blocker) Moving to marketplace-side Phase 5 work on the `development_mik02` line (pair endpoint + pending-nodes UI). This is pure marketplace code, direct-push, and can progress without waiting for any upstream merge. The `hosting.is_farm_owner` / `node_get_by_identity` verification inside the pair endpoint ships behind a feature flag (`HOSTING_VERIFICATION=mock`) that flips to `real` once devnet has the contract.
mik-tf commented 2026-04-11 02:22:22 +00:00
Author
Member
Copy link

Session wrap — 2026-04-10 (Phase 1 live + Phase 3 upstream ready)

What landed in development (merged)

Repo Commit PR
projectmycelium_marketplace_backend 9244431 #1 squash-merged
projectmycelium_marketplace_frontend 5da2f80 #1 squash-merged

Phase 1 crude bridge (add node from mycelium IP) is now on development in both marketplace repos. Already running on dev-app.projectmycelium.org as :development_mik02 container images.

What is open upstream and awaiting maintainer review

Repo Artifact State
lhumina_code/hero_compute PR #91 — heartbeat preserve fix + Phase 3 Steps 1-4 CI green (Test, Format, Clippy all pass)
lhumina_code/hero_ledger PR #45 — add hosting to deploy_services.rhai CI runner-side auth bug (unrelated to diff)
lhumina_code/hero_ledger issue #44 — tracks hosting deployment gap Open

@mahmoud — hero_compute PR #91 is waiting on you (CI green, Phase 3 Steps 1-4). @scott — hero_ledger PR #45 + issue #44 are waiting on you (hosting contract devnet deploy + runner auth fix). No urgency on any of it; session notes below explain why none of it blocks our Phase 5 marketplace work.

Issues closed

Dev VM state

Everything on dev-app.projectmycelium.org is working end-to-end:

  • Backend + frontend running :development_mik02 images via docker-compose.override.yml pin on the dev VM
  • hero_compute stack (server + explorer TCP:9002 + UI) running under three clean new systemd units (hero-compute-server.service, hero-compute-explorer.service, hero-compute-ui.service), replacing the broken hero-compute.service which ran hero_compute --start --mode local that fails with Symbolic link loop (os error 40) on this host
  • hero-compute-server.service unit has EXPLORER_ADDRESSES=/root/hero/var/sockets/hero_compute_explorer.sock + MYCELIUM_IP=46a:52b7:d2c2:4416:ff0f:5892:d922:50dc in Environment=, so the node now heartbeats with its correct IP
  • Real node 0001 (devpmmarketplace, mycelium 46a:52b7:d2c2:4416:ff0f:5892:d922:50dc) visible and online via marketplace Phase 1 endpoint, imported as FarmNode 01ce with grid_data.compute_node_sid populated for Phase 5 pairing linkage

Tests on dev-app

444 pass, 7 pre-existing non-regression failures (2 DEMO_KYC onboarding artifacts that predate the flag, 5 MCP socket tests that require a local marketplace instance). None caused by Phase 1 or the scaling work.

Phase 3 status (upstream hero_compute)

Steps 1-4 of the Phase 3 spec are in PR #91 as 5 feature commits + 2 style fixups:

# Commit Scope LOC
defensive fix ef17fe8 heartbeat mycelium_ip preserve +73
1 bd04f49 identity.rs: persistent ed25519 + NEAR implicit account +348
2 47b4f33 main.rs: load identity at startup, log pubkey +14
3 a92c6ce heartbeat_sender: sign canonical payload +215/-3
4 38b1c8e explorer: verify signature + ALLOW_LEGACY_HEARTBEATS grace period +543/-8
style fcf82c1 54eb0ac cargo fmt + clippy dead_code allows +38/-46

Tests: 29 server + 15 explorer = 44 unit tests green. Includes red→green verification of both the heartbeat preserve bug and the signature tamper detection.

Step 5 (first-boot registration hook) intentionally deferred to a separate follow-up PR on hero_compute once hero_ledger PR #45 merges and hosting.dev.hero exists on devnet. Rationale:

  1. Drags in the full near-jsonrpc-client / near-primitives / near-crypto tree — heavy deps orthogonal to identity + signing.
  2. Nothing to E2E-verify against until devnet has the hosting contract.
  3. The Phase 3.5 first-boot hook stacks cleanly on top of Steps 1-4; the public NodeIdentity API it will consume (public_key_bytes, sign_raw, verify_raw, verify_hashed) already exists on the branch with #[allow(dead_code)] markers and doc comments explicitly pointing at Phase 3.5.

Phase status table (post-session)

Phase Status
0 Prep + spec + issue ✅ Done
1 Crude bridge (add node from mycelium IP) ✅ Shipped to development on both marketplace repos + live on dev VM
2 Gateway hosting client methods 🔄 Deferred to Phase 2.5 parallel workstream
3.0 NEAR implicit accounts checkpoint ✅ Passed + revealed hosting contract deployment gap (hero_ledger#44, PR #45)
3 Node crypto identity in hero_compute 🟡 Steps 1-4 ready for review (hero_compute#91), Step 5 deferred to Phase 3.5 follow-up
3.5 First-boot registration hook ⏳ Ready to start once hosting is on devnet
4 mkt-prepare-usb tool ⏳ Ready to start (marketplace-side, no upstream dep)
5 Auto-pairing endpoint + pending UI ⏳ Next session priority (marketplace-side, no upstream dep)
6 Pricing templates Stretch
7 FarmNode → cache Stretch
8 Regional explorers Deferred

Next session focus

Phase 5 marketplace-side work: POST /api/nodes/pair endpoint + pending-nodes farmer dashboard UI + publish action + shell integration tests. Direct-push on development_mik02 per the marketplace workflow. No upstream blockers — hosting verification ships behind a HOSTING_VERIFICATION=mock|real feature flag that can flip to real once devnet catches up.

## Session wrap — 2026-04-10 (Phase 1 live + Phase 3 upstream ready) ### What landed in development (merged) | Repo | Commit | PR | |---|---|---| | projectmycelium_marketplace_backend | `9244431` | [#1](https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_backend/pulls/1) squash-merged | | projectmycelium_marketplace_frontend | `5da2f80` | [#1](https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_frontend/pulls/1) squash-merged | Phase 1 crude bridge (add node from mycelium IP) is now on `development` in both marketplace repos. Already running on `dev-app.projectmycelium.org` as `:development_mik02` container images. ### What is open upstream and awaiting maintainer review | Repo | Artifact | State | |---|---|---| | `lhumina_code/hero_compute` | [PR #91](https://forge.ourworld.tf/lhumina_code/hero_compute/pulls/91) — heartbeat preserve fix + Phase 3 Steps 1-4 | CI green (Test, Format, Clippy all pass) | | `lhumina_code/hero_ledger` | [PR #45](https://forge.ourworld.tf/lhumina_code/hero_ledger/pulls/45) — add hosting to `deploy_services.rhai` | CI runner-side auth bug (unrelated to diff) | | `lhumina_code/hero_ledger` | [issue #44](https://forge.ourworld.tf/lhumina_code/hero_ledger/issues/44) — tracks hosting deployment gap | Open | @mahmoud — hero_compute PR #91 is waiting on you (CI green, Phase 3 Steps 1-4). @scott — hero_ledger PR #45 + issue #44 are waiting on you (hosting contract devnet deploy + runner auth fix). No urgency on any of it; session notes below explain why none of it blocks our Phase 5 marketplace work. ### Issues closed - [#71 MOS bare-metal + full stack integration](https://forge.ourworld.tf/mycelium_code/home/issues/71) — closed with full Phase 1 deploy evidence and test totals. ### Dev VM state Everything on `dev-app.projectmycelium.org` is working end-to-end: - Backend + frontend running `:development_mik02` images via `docker-compose.override.yml` pin on the dev VM - hero_compute stack (server + explorer TCP:9002 + UI) running under three clean new systemd units (`hero-compute-server.service`, `hero-compute-explorer.service`, `hero-compute-ui.service`), replacing the broken `hero-compute.service` which ran `hero_compute --start --mode local` that fails with `Symbolic link loop (os error 40)` on this host - `hero-compute-server.service` unit has `EXPLORER_ADDRESSES=/root/hero/var/sockets/hero_compute_explorer.sock` + `MYCELIUM_IP=46a:52b7:d2c2:4416:ff0f:5892:d922:50dc` in `Environment=`, so the node now heartbeats with its correct IP - Real node `0001` (`devpmmarketplace`, mycelium `46a:52b7:d2c2:4416:ff0f:5892:d922:50dc`) visible and online via marketplace Phase 1 endpoint, imported as FarmNode `01ce` with `grid_data.compute_node_sid` populated for Phase 5 pairing linkage ### Tests on dev-app 444 pass, 7 pre-existing non-regression failures (2 DEMO_KYC onboarding artifacts that predate the flag, 5 MCP socket tests that require a local marketplace instance). None caused by Phase 1 or the scaling work. ### Phase 3 status (upstream hero_compute) Steps 1-4 of the Phase 3 spec are in PR #91 as 5 feature commits + 2 style fixups: | # | Commit | Scope | LOC | |---|---|---|---| | defensive fix | `ef17fe8` | heartbeat mycelium_ip preserve | +73 | | 1 | `bd04f49` | identity.rs: persistent ed25519 + NEAR implicit account | +348 | | 2 | `47b4f33` | main.rs: load identity at startup, log pubkey | +14 | | 3 | `a92c6ce` | heartbeat_sender: sign canonical payload | +215/-3 | | 4 | `38b1c8e` | explorer: verify signature + ALLOW_LEGACY_HEARTBEATS grace period | +543/-8 | | style | `fcf82c1` `54eb0ac` | cargo fmt + clippy dead_code allows | +38/-46 | Tests: 29 server + 15 explorer = **44 unit tests green**. Includes red→green verification of both the heartbeat preserve bug and the signature tamper detection. **Step 5 (first-boot registration hook) intentionally deferred** to a separate follow-up PR on hero_compute once hero_ledger PR #45 merges and `hosting.dev.hero` exists on devnet. Rationale: 1. Drags in the full `near-jsonrpc-client` / `near-primitives` / `near-crypto` tree — heavy deps orthogonal to identity + signing. 2. Nothing to E2E-verify against until devnet has the hosting contract. 3. The Phase 3.5 first-boot hook stacks cleanly on top of Steps 1-4; the public `NodeIdentity` API it will consume (`public_key_bytes`, `sign_raw`, `verify_raw`, `verify_hashed`) already exists on the branch with `#[allow(dead_code)]` markers and doc comments explicitly pointing at Phase 3.5. ### Phase status table (post-session) | Phase | Status | |---|---| | **0** Prep + spec + issue | ✅ Done | | **1** Crude bridge (add node from mycelium IP) | ✅ **Shipped to development on both marketplace repos + live on dev VM** | | **2** Gateway hosting client methods | 🔄 Deferred to Phase 2.5 parallel workstream | | **3.0** NEAR implicit accounts checkpoint | ✅ Passed + revealed hosting contract deployment gap ([hero_ledger#44](https://forge.ourworld.tf/lhumina_code/hero_ledger/issues/44), [PR #45](https://forge.ourworld.tf/lhumina_code/hero_ledger/pulls/45)) | | **3** Node crypto identity in hero_compute | 🟡 **Steps 1-4 ready for review** ([hero_compute#91](https://forge.ourworld.tf/lhumina_code/hero_compute/pulls/91)), Step 5 deferred to Phase 3.5 follow-up | | **3.5** First-boot registration hook | ⏳ Ready to start once hosting is on devnet | | **4** mkt-prepare-usb tool | ⏳ Ready to start (marketplace-side, no upstream dep) | | **5** Auto-pairing endpoint + pending UI | ⏳ Next session priority (marketplace-side, no upstream dep) | | **6** Pricing templates | Stretch | | **7** FarmNode → cache | Stretch | | **8** Regional explorers | Deferred | ### Next session focus Phase 5 marketplace-side work: `POST /api/nodes/pair` endpoint + pending-nodes farmer dashboard UI + publish action + shell integration tests. Direct-push on `development_mik02` per the marketplace workflow. No upstream blockers — hosting verification ships behind a `HOSTING_VERIFICATION=mock|real` feature flag that can flip to real once devnet catches up.
mik-tf commented 2026-04-11 12:46:44 +00:00
Author
Member
Copy link

Session 3 docs refresh — vision.md + e2e_checklist.md

Two follow-up docs changes after the earlier session-wrap comment, both pushed to development_mik02 on the deploy repo:

New docs/vision.md (272 lines, commit 2677346)

North-star doc that captures the target state in human-readable terms. Fills a gap in the existing docs:

  • architecture.md describes current state (what is running today)
  • DESIGN.md is SPA-focused (client-owned identity, signature auth, payment system)
  • complete_ux_map.md is exhaustive route-level SSOT
  • scaling_architecture.md is the phase-by-phase delivery plan
  • vision.md is the "read this first" doc that explains what we are building, why, and what it feels like end to end

Contents:

  1. Vision in one sentence + six guiding principles
  2. Step-by-step target farmer UX (one-time setup + per-node onboarding + publishing + ongoing)
  3. Step-by-step target user UX (setup + renting + during + on-expiry)
  4. Trust model table (every trust question → who verifies → how)
  5. Four-store data model — the hosting contract + marketplace contract + marketplace backend OSIS + hero_compute explorer, what lives where, why, with a write-sequence diagram showing how a first-boot node lands data in all four stores
  6. Why ownership and commercial availability are split across two on-chain contracts instead of one big registry
  7. Federation / white-label story
  8. The 45-second claim unpacked — wall-clock human effort per node
  9. Phase 1 reality vs target state, per aspect
  10. Cross-reference table to every other doc

docs/e2e_checklist.md refresh (commit ae10141, +112/-20)

Status refresh for everything that moved in Session 3, plus 28 new rows for the scaling architecture initiative:

New status marker: ⏳ = "code complete on branch, awaiting upstream maintainer merge". Used for all Phase 3 Steps 1-4 rows that are on hero_compute#91 but not yet merged.

Status refreshes (existing rows): #188 hero_compute heartbeat ⚠️ → ✅, #193/#194/#195 frontend pages ❌ → ✅ (stale markers from Session 2 work), F13/F14 node health + auto-heartbeat ⚠️/❌ → ✅, P14 test count 354 → 444.

New rows:

  • #196 Phase 1 add-node-from-explorer (shipped in this session)
  • F16-F26 (11 new farmer rows) — SPA-side NEAR vault, hosting.farm_create, mkt-prepare-usb, first-boot node_register, signed heartbeats, POST /api/nodes/pair, pending nodes UI, publish action, auto-publish toggle, on-chain SPORE wallet, 45-second onboarding target
  • U23-U25 (3 new user rows) — on-chain proof chain inspector, createReservation clarity row, on-chain tokens.balance wallet
  • P15-P27 (13 new platform rows) — hosting contract deployment gap, node ed25519 identity (⏳), startup integration (⏳), canonical payload (⏳), explorer verification (⏳), first-boot hook, mkt-prepare-usb CLI, pair endpoint, HOSTING_VERIFICATION flag, publish endpoint, pending listing, marketplace as cache, multi-marketplace federation

Every ⏳ row traces to a specific commit on hero_compute#91. Every upstream-blocked ❌ row cross-references hero_ledger #44/#45. Both docs cross-reference each other so readers can navigate between "what is this?" (vision.md) and "where are we at?" (e2e_checklist.md).

Maintainer attribution cleanup

Fixed @scott → @mahmoud in three places where hero_compute review was incorrectly tagged to the hero_ledger maintainer:

  • Comment 19095 "Awaiting @scott review" on the Phase 3 update → @mahmoud
  • Comment 19104 session-wrap "@scott — all three items" → split into "@mahmoud — hero_compute#91" + "@scott — hero_ledger#45/#44"
  • prompt.md phase status table + upstream PR table

New ping comment on hero_compute#91 explicitly tagging @mahmoud since the PR description did not mention them (comment 19113).

Session end

Nothing more to ship in this session. Next session starts on Phase 5 marketplace-side work: POST /api/nodes/pair backend endpoint + publish endpoint + pending nodes UI, all direct-pushed to development_mik02 per the marketplace workflow. HOSTING_VERIFICATION=mock feature flag unblocks this entirely from the hero_ledger hosting deployment delay.

## Session 3 docs refresh — vision.md + e2e_checklist.md Two follow-up docs changes after the earlier session-wrap comment, both pushed to `development_mik02` on the deploy repo: ### New `docs/vision.md` (272 lines, commit `2677346`) North-star doc that captures the target state in human-readable terms. Fills a gap in the existing docs: - `architecture.md` describes current state (what is running today) - `DESIGN.md` is SPA-focused (client-owned identity, signature auth, payment system) - `complete_ux_map.md` is exhaustive route-level SSOT - `scaling_architecture.md` is the phase-by-phase delivery plan - `vision.md` is the "read this first" doc that explains **what** we are building, **why**, and **what it feels like** end to end Contents: 1. Vision in one sentence + six guiding principles 2. Step-by-step target farmer UX (one-time setup + per-node onboarding + publishing + ongoing) 3. Step-by-step target user UX (setup + renting + during + on-expiry) 4. Trust model table (every trust question → who verifies → how) 5. **Four-store data model** — the `hosting` contract + `marketplace` contract + marketplace backend OSIS + hero_compute explorer, what lives where, why, with a write-sequence diagram showing how a first-boot node lands data in all four stores 6. Why ownership and commercial availability are split across two on-chain contracts instead of one big registry 7. Federation / white-label story 8. The 45-second claim unpacked — wall-clock human effort per node 9. Phase 1 reality vs target state, per aspect 10. Cross-reference table to every other doc ### `docs/e2e_checklist.md` refresh (commit `ae10141`, +112/-20) Status refresh for everything that moved in Session 3, plus 28 new rows for the scaling architecture initiative: **New status marker**: `⏳` = "code complete on branch, awaiting upstream maintainer merge". Used for all Phase 3 Steps 1-4 rows that are on hero_compute#91 but not yet merged. **Status refreshes (existing rows)**: #188 hero_compute heartbeat `⚠️ → ✅`, #193/#194/#195 frontend pages `❌ → ✅` (stale markers from Session 2 work), F13/F14 node health + auto-heartbeat `⚠️/❌ → ✅`, P14 test count `354 → 444`. **New rows**: - `#196` Phase 1 add-node-from-explorer (shipped in this session) - **F16-F26** (11 new farmer rows) — SPA-side NEAR vault, `hosting.farm_create`, `mkt-prepare-usb`, first-boot `node_register`, signed heartbeats, `POST /api/nodes/pair`, pending nodes UI, publish action, auto-publish toggle, on-chain SPORE wallet, 45-second onboarding target - **U23-U25** (3 new user rows) — on-chain proof chain inspector, `createReservation` clarity row, on-chain `tokens.balance` wallet - **P15-P27** (13 new platform rows) — hosting contract deployment gap, node ed25519 identity (⏳), startup integration (⏳), canonical payload (⏳), explorer verification (⏳), first-boot hook, mkt-prepare-usb CLI, pair endpoint, `HOSTING_VERIFICATION` flag, publish endpoint, pending listing, marketplace as cache, multi-marketplace federation Every `⏳` row traces to a specific commit on hero_compute#91. Every upstream-blocked `❌` row cross-references hero_ledger #44/#45. Both docs cross-reference each other so readers can navigate between "what is this?" (`vision.md`) and "where are we at?" (`e2e_checklist.md`). ### Maintainer attribution cleanup Fixed `@scott` → `@mahmoud` in three places where hero_compute review was incorrectly tagged to the hero_ledger maintainer: - Comment 19095 "Awaiting @scott review" on the Phase 3 update → `@mahmoud` - Comment 19104 session-wrap "@scott — all three items" → split into "@mahmoud — hero_compute#91" + "@scott — hero_ledger#45/#44" - `prompt.md` phase status table + upstream PR table New ping comment on hero_compute#91 explicitly tagging @mahmoud since the PR description did not mention them (comment 19113). ### Session end Nothing more to ship in this session. Next session starts on Phase 5 marketplace-side work: `POST /api/nodes/pair` backend endpoint + publish endpoint + pending nodes UI, all direct-pushed to `development_mik02` per the marketplace workflow. `HOSTING_VERIFICATION=mock` feature flag unblocks this entirely from the hero_ledger hosting deployment delay.
mik-tf commented 2026-04-11 14:09:04 +00:00
Author
Member
Copy link

Phase 5 marketplace-side shipped — signed pairing, pending UI, publish flow

2026-04-11 session. Full marketplace-side Phase 5 is code-complete on development_mik02 across all three repos, tested via unit + shell integration, awaiting remote deploy + verification on dev-app.projectmycelium.org.

What shipped

Backend (projectmycelium_marketplace_backend @ c0d21ff — dade02a Phase 5 base + c0d21ff UserPersistence fix after first integration run)

New module src/services/pairing.rs (+438 LOC, 12 unit tests):

  • canonical_pair_payload — byte-stable sorted-key JSON via BTreeMap<&str, serde_json::Value> + serde_json::to_vec. Matches the Phase 3 heartbeat canonicalization pattern from hero_compute_server::heartbeat_sender exactly. The nested specs object is also built as an explicit BTreeMap so ordering is guaranteed regardless of serde_json feature flags later.
  • verify_pair_signature — sha256(canonical) then ed25519-verify against the 32-byte digest. Plain ed25519 over the digest, not ed25519ph — matches hero_compute::identity::Identity::sign_hashed / verify_hashed. Base64 signature encoding on the wire.
  • parse_implicit_account — hex-decode + validate 64 chars, rejects named-form inputs (fred.dev.hero) with a clear error pointing at docs/vision.md §11. Used for both node_account and farmer_account.
  • check_timestamp_skew — ±300 s replay window.
  • PairError — thiserror enum mapped to HTTP 400 for malformed inputs, 401 for signature/timestamp failures.

New controller src/axum_app/controllers/nodes.rs (+560 LOC, three handlers):

  • POST /api/nodes/pair (unauth, signature-authenticated). Decodes + validates both accounts, checks timestamp, rebuilds canonical bytes, verifies signature, enforces HOSTING_VERIFICATION feature flag (mock default; real returns 501 pending hero_ledger#45 + gateway client hosting.rs), resolves farmer_account → marketplace user via auth.find_by_public_key, enforces idempotency (pending replay → 200, already published → 409), builds NodeCreationData via the same builder Phase 1 uses, calls resource_provider.add_node, patches grid_data with pairing_state=pending + all identity fields + paired_at + best-effort compute_node_sid from explorer, persists, logs NodeAdded activity, returns 201.
  • GET /api/dashboard/nodes/pending (session auth). Loads farmer's nodes, filters by grid_data.pairing_state == "pending", returns {count, nodes}.
  • POST /api/dashboard/nodes/:id/publish (session auth). Enforces ownership + pending state, builds the same marketplace.listing_create body shape Phase 1 uses, calls ledger_gateway.marketplace_create_listing, patches grid_data pending→published + ledger_listing_id + published_at + hourly_spore, returns 200. Gateway failures return 502 with no state mutation.

Routes wired in src/axum_app/routes.rs. Phase 1's /api/dashboard/nodes/from-explorer is retained as a fallback alongside Phase 5's pair endpoint — the original scaling_architecture.md spec called for deleting it, but deleting now would leave farmers with no working add-node path since Phase 3.5 (the first-boot hook that will auto-call /api/nodes/pair) isn't live yet.

Data layer wrinkle found during first dev-VM integration run: LocalResourceProviderManager (the OSIS-backed backend the dev VM uses via local_with_ledger_and_compute) drops free-form grid_data entirely — osis_node_to_farm_node hardcodes grid_data: None on read, and update_node only extracts three typed fields (ledger_listing_id, compute_node_sid, ledger_account_id) from the grid_data JSON it accepts, discarding everything else. So writing pairing_state through update_node looked like it worked but vanished on every subsequent get_nodes call.

Fix in c0d21ff: use UserPersistence (the per-user JSON file store) as the canonical store for pairing_state and all other Phase 5 grid_data fields. UserPersistence serializes the full FarmNode struct so it preserves arbitrary grid_data. The pair handler now pushes the FarmNode into UserPersistence.nodes after add_node, list_pending reads from UserPersistence::load_user_data(email).nodes, and publish does the same. update_node is still called as a follow-up to mirror compute_node_sid + ledger_listing_id into OSIS typed fields so the rental flow (which reads from OSIS) keeps working. Phase 7 flips this around when the marketplace backend becomes a pure cache over the on-chain hosting contract — at that point pairing_state disappears entirely because on-chain state is authoritative.

Frontend (projectmycelium_marketplace_frontend @ 42396e9)

New file src/pages/dashboard_pending_nodes.rs — PendingNodesSection component rendered above "My Nodes" on the farmer dashboard:

  • Card grid (dioxus-bootstrap-css Card) showing hostname, CPU/RAM/disk, truncated mycelium IPv6, truncated node implicit account + farm_id, paired_at timestamp, and a "mock verification mode" hint when HOSTING_VERIFICATION=mock (the default today).
  • Publish button opens a dioxus-bootstrap-css Modal with four hardcoded pricing tiers matching vision.md §4 step 7 (nano/basic/standard/premium → 20/60/125/250 SPORE/hour). Selected tier's hourly_spore is POSTed to /api/dashboard/nodes/:id/publish. On success, both the pending list and the main nodes table refresh, and a success toast fires.
  • Auto-hides when the pending list is empty — no "0 pending" empty state cluttering the dashboard for farmers who haven't paired any nodes yet.
  • NO hand-rolled modals or toggles — all interactivity via dioxus-bootstrap-css components per the project convention.

Deploy (projectmycelium_marketplace_deploy @ a603e1d)

  • 5de3443 — docs/vision.md §11 "Account types — implicit vs named" hard rule codified: farmers, users, and nodes are always 64-char hex implicit accounts; named accounts (hosting.dev.hero etc.) are reserved for deploy-owned system identities only. Rationale included (self-sovereignty, no gatekeeper at onboarding, federation by construction) so future readers understand the constraint rather than just follow it.
  • a603e1d — tests/pairing_integration.sh (8 scenarios, ~490 lines) that fakes a hero_compute node using Python's cryptography package, signs canonical pair payloads byte-identically to the backend's Rust verifier, and exercises the full farmer-side Phase 5 flow end-to-end. Makefile test-pairing target + slotted into test-all.

Coverage:

  1. Farmer SPA keypair registration (public_key flow, not password) so find_by_public_key has something to match
  2. Valid signed pair → 201
  3. GET /api/dashboard/nodes/pending contains the paired node
  4. POST /api/dashboard/nodes/:id/publish with hourly_spore → 200 (or 502 if ledger unreachable, tolerated in environment-specific envs)
  5. Idempotent replay → 409 (already published) or 200 (still pending idempotent, ledger-unreachable path)
  6. Tampered base64 signature → 401
  7. Timestamp 10 minutes in the past → 401
  8. farmer_account = "fred.dev.hero" (named form) → 400 with the "implicit accounts only" error

The canonical payload contract is locked in by a golden unit test in services::pairing (canonical_payload_matches_python_json_dumps_sorted) that compares the Rust canonical_pair_payload output byte-for-byte against the exact string that json.dumps(sort_keys=True, separators=(",",":"), ensure_ascii=False) produces in the Python signer. If either side drifts, CI fails before any shell test runs.

Test results

Local unit tests:

  • cargo check --lib on backend: clean, 0 errors
  • cargo test --lib: 37 passed, 0 failed (25 prior + 12 pairing unit tests including the Python canonical-bytes golden cross-check)
  • cargo check --target wasm32-unknown-unknown on frontend: clean, 0 errors
  • bash -n tests/pairing_integration.sh: syntax OK

Remote integration against dev-app.projectmycelium.org:

  • tests/pairing_integration.sh: 11/11 passed, 0 failed
    • Test 1: Farmer SPA keypair registration → 200
    • Test 2: Valid signed pair → 201
    • Test 3: GET pending list contains the paired node → count=1
    • Test 4: Publish endpoint reachable (returns 502 from hero_ledger — see "out of scope" below; state correctly preserved on gateway failure)
    • Test 5: Idempotent replay → 200 with idempotent_replay=true
    • Test 6: Tampered base64 signature → 401
    • Test 7: Stale timestamp (10 min old) → 401
    • Test 8: Named form farmer_account="fred.dev.hero" → 400 with "implicit accounts only" error

Regression (focused — no touches to these layers but verifying blast radius is zero):

  • api_smoke.sh → 27/27 passed
  • provider_integration.sh → 34/34 passed
  • farmer_integration.sh → 34/34 passed

Zero regressions on the three most-at-risk suites. Full make test-all sweep not re-run because the Phase 5 changes are additive (new module, new controller file, three new routes, one new frontend component) with no edits to existing handlers.

Out of scope for Phase 5 (follow-up)

The publish endpoint's 502 is not a Phase 5 bug. Dev VM backend logs show:

[nodes::publish] ledger listing creation failed for node 01cy:
RPC error (-32602): Invalid params: V3 listings must use
marketplace.createV3Listing with TF Chain ownership proof

The hero_ledger marketplace contract has evolved since Phase 1 landed — it now requires marketplace.createV3Listing with on-chain ownership proof, and rejects the legacy marketplace.create_listing call shape that both Phase 1's add_node_from_explorer and my Phase 5 publish handler build. Both handlers need to migrate to the v3 method in a follow-up session. Phase 5's own logic correctly: signature-verifies, creates the FarmNode in pending state, filters on pending, calls the (now-wrong) method, returns 502 without state mutation on gateway failure — exactly the specified contract.

What's deferred (explicit)

  1. F24 "Auto-publish toggle" — Requires a user preference field + conditional publish-on-pair path in the pair handler. Cut from this session to keep Phase 5 focused on the review-then-approve UX that matches vision.md §3 steps 11-13. Follow-up session.
  2. HOSTING_VERIFICATION=real — Handler stub returns 501 Not Implemented with a clear message pointing at the two unblock steps:
    • hero_ledger#45 merge → hosting.dev.hero deploy on devnet
    • hero_ledger gateway client gets hosting.rs methods (Phase 2.5 parallel workstream, no upstream dep)
      Flipping to real is a ~20 LOC diff in nodes::pair once those are ready.
  3. PricingTemplate — Phase 6. The publish endpoint currently accepts {hourly_spore: u64}; the swap to {pricing_template_id} is a one-line body field change when the OSIS root object lands.
  4. Deletion of Phase 1 /api/dashboard/nodes/from-explorer — Retained as a fallback until Phase 3.5 ships the node-side caller. Delete in the same PR that lands Phase 3.5.
  5. Playwright E2E coverage for the new UI — API is covered by pairing_integration.sh; a browser regression belongs in the same session that lands Phase 3.5 real-node pairing, when there'll be a deterministic fixture path to drive it.

Phase status after this session

Phase Status Owner
1 — Crude bridge ✅ Live on dev VM (retained as fallback) mik-tf
2 — Gateway hosting methods 🔄 Deferred to Phase 2.5 parallel workstream —
3 — Node crypto identity (hero_compute) 🟡 Code complete on #91, awaiting @mahmoud review @mahmoud
3.5 — First-boot registration hook ⏳ Blocked on hero_ledger#45 merge + devnet hosting deploy @scott (deploy), @mahmoud (hook)
4 — mkt-prepare-usb ⏳ Ready to start, no upstream dep mik-tf
5 — Auto-pairing + pending UI (marketplace side) ✅ Shipped 2026-04-11 on development_mik02 mik-tf
6 — Pricing templates Stretch — one-line swap in publish endpoint when templates land —
7 — FarmNode → pure cache Stretch — requires 3.5 live + 5 merged —

Upstream PR status (unchanged this session — we do not self-merge upstream)

Per the upstream-PR policy: marketplace repos ship fast via direct push to development_mik02, upstream hero_* and mos_* repos always go via PRs and the maintainer merges.

Signed-off-by: mik-tf

## Phase 5 marketplace-side shipped — signed pairing, pending UI, publish flow 2026-04-11 session. Full marketplace-side Phase 5 is code-complete on `development_mik02` across all three repos, tested via unit + shell integration, awaiting remote deploy + verification on `dev-app.projectmycelium.org`. ### What shipped **Backend** (`projectmycelium_marketplace_backend` @ [`c0d21ff`](https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_backend/commit/c0d21ff) — [`dade02a`](https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_backend/commit/dade02a) Phase 5 base + [`c0d21ff`](https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_backend/commit/c0d21ff) UserPersistence fix after first integration run) New module `src/services/pairing.rs` (+438 LOC, 12 unit tests): - `canonical_pair_payload` — byte-stable sorted-key JSON via `BTreeMap<&str, serde_json::Value>` + `serde_json::to_vec`. Matches the Phase 3 heartbeat canonicalization pattern from `hero_compute_server::heartbeat_sender` exactly. The nested `specs` object is also built as an explicit `BTreeMap` so ordering is guaranteed regardless of serde_json feature flags later. - `verify_pair_signature` — `sha256(canonical)` then ed25519-verify against the 32-byte digest. Plain ed25519 over the digest, **not** ed25519ph — matches `hero_compute::identity::Identity::sign_hashed` / `verify_hashed`. Base64 signature encoding on the wire. - `parse_implicit_account` — hex-decode + validate 64 chars, **rejects named-form inputs** (`fred.dev.hero`) with a clear error pointing at `docs/vision.md §11`. Used for both `node_account` and `farmer_account`. - `check_timestamp_skew` — ±300 s replay window. - `PairError` — thiserror enum mapped to HTTP 400 for malformed inputs, 401 for signature/timestamp failures. New controller `src/axum_app/controllers/nodes.rs` (+560 LOC, three handlers): - **`POST /api/nodes/pair`** (unauth, signature-authenticated). Decodes + validates both accounts, checks timestamp, rebuilds canonical bytes, verifies signature, enforces `HOSTING_VERIFICATION` feature flag (mock default; real returns 501 pending hero_ledger#45 + gateway client `hosting.rs`), resolves `farmer_account` → marketplace user via `auth.find_by_public_key`, enforces idempotency (pending replay → 200, already published → 409), builds `NodeCreationData` via the same builder Phase 1 uses, calls `resource_provider.add_node`, patches `grid_data` with `pairing_state=pending` + all identity fields + `paired_at` + best-effort `compute_node_sid` from explorer, persists, logs `NodeAdded` activity, returns 201. - **`GET /api/dashboard/nodes/pending`** (session auth). Loads farmer's nodes, filters by `grid_data.pairing_state == "pending"`, returns `{count, nodes}`. - **`POST /api/dashboard/nodes/:id/publish`** (session auth). Enforces ownership + pending state, builds the same `marketplace.listing_create` body shape Phase 1 uses, calls `ledger_gateway.marketplace_create_listing`, patches `grid_data` pending→published + `ledger_listing_id` + `published_at` + `hourly_spore`, returns 200. Gateway failures return 502 with no state mutation. Routes wired in `src/axum_app/routes.rs`. **Phase 1's `/api/dashboard/nodes/from-explorer` is retained** as a fallback alongside Phase 5's pair endpoint — the original scaling_architecture.md spec called for deleting it, but deleting now would leave farmers with no working add-node path since Phase 3.5 (the first-boot hook that will auto-call `/api/nodes/pair`) isn't live yet. **Data layer wrinkle found during first dev-VM integration run**: `LocalResourceProviderManager` (the OSIS-backed backend the dev VM uses via `local_with_ledger_and_compute`) drops free-form `grid_data` entirely — `osis_node_to_farm_node` hardcodes `grid_data: None` on read, and `update_node` only extracts three typed fields (`ledger_listing_id`, `compute_node_sid`, `ledger_account_id`) from the `grid_data` JSON it accepts, discarding everything else. So writing `pairing_state` through `update_node` looked like it worked but vanished on every subsequent `get_nodes` call. Fix in [`c0d21ff`](https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_backend/commit/c0d21ff): use `UserPersistence` (the per-user JSON file store) as the canonical store for pairing_state and all other Phase 5 grid_data fields. `UserPersistence` serializes the full `FarmNode` struct so it preserves arbitrary grid_data. The pair handler now pushes the FarmNode into `UserPersistence.nodes` after `add_node`, `list_pending` reads from `UserPersistence::load_user_data(email).nodes`, and `publish` does the same. `update_node` is still called as a follow-up to mirror `compute_node_sid` + `ledger_listing_id` into OSIS typed fields so the rental flow (which reads from OSIS) keeps working. Phase 7 flips this around when the marketplace backend becomes a pure cache over the on-chain hosting contract — at that point `pairing_state` disappears entirely because on-chain state is authoritative. **Frontend** (`projectmycelium_marketplace_frontend` @ [`42396e9`](https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_frontend/commit/42396e9)) New file `src/pages/dashboard_pending_nodes.rs` — `PendingNodesSection` component rendered above "My Nodes" on the farmer dashboard: - Card grid (dioxus-bootstrap-css `Card`) showing hostname, CPU/RAM/disk, truncated mycelium IPv6, truncated node implicit account + farm_id, `paired_at` timestamp, and a "mock verification mode" hint when `HOSTING_VERIFICATION=mock` (the default today). - Publish button opens a dioxus-bootstrap-css `Modal` with four hardcoded pricing tiers matching vision.md §4 step 7 (`nano/basic/standard/premium` → 20/60/125/250 SPORE/hour). Selected tier's `hourly_spore` is POSTed to `/api/dashboard/nodes/:id/publish`. On success, both the pending list and the main nodes table refresh, and a success toast fires. - Auto-hides when the pending list is empty — no "0 pending" empty state cluttering the dashboard for farmers who haven't paired any nodes yet. - NO hand-rolled modals or toggles — all interactivity via dioxus-bootstrap-css components per the project convention. **Deploy** (`projectmycelium_marketplace_deploy` @ [`a603e1d`](https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_deploy/commit/a603e1d)) - [`5de3443`](https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_deploy/commit/5de3443) — `docs/vision.md §11 "Account types — implicit vs named"` hard rule codified: farmers, users, and nodes are always 64-char hex implicit accounts; named accounts (`hosting.dev.hero` etc.) are reserved for deploy-owned system identities only. Rationale included (self-sovereignty, no gatekeeper at onboarding, federation by construction) so future readers understand the constraint rather than just follow it. - [`a603e1d`](https://forge.ourworld.tf/mycelium_code/projectmycelium_marketplace_deploy/commit/a603e1d) — `tests/pairing_integration.sh` (8 scenarios, ~490 lines) that fakes a hero_compute node using Python's `cryptography` package, signs canonical pair payloads byte-identically to the backend's Rust verifier, and exercises the full farmer-side Phase 5 flow end-to-end. Makefile `test-pairing` target + slotted into `test-all`. Coverage: 1. Farmer SPA keypair registration (`public_key` flow, not password) so `find_by_public_key` has something to match 2. Valid signed pair → 201 3. `GET /api/dashboard/nodes/pending` contains the paired node 4. `POST /api/dashboard/nodes/:id/publish` with `hourly_spore` → 200 (or 502 if ledger unreachable, tolerated in environment-specific envs) 5. Idempotent replay → 409 (already published) or 200 (still pending idempotent, ledger-unreachable path) 6. Tampered base64 signature → 401 7. Timestamp 10 minutes in the past → 401 8. `farmer_account = "fred.dev.hero"` (named form) → 400 with the "implicit accounts only" error The canonical payload contract is locked in by a golden unit test in `services::pairing` (`canonical_payload_matches_python_json_dumps_sorted`) that compares the Rust `canonical_pair_payload` output byte-for-byte against the exact string that `json.dumps(sort_keys=True, separators=(",",":"), ensure_ascii=False)` produces in the Python signer. If either side drifts, CI fails before any shell test runs. ### Test results **Local unit tests:** - `cargo check --lib` on backend: clean, 0 errors - `cargo test --lib`: **37 passed, 0 failed** (25 prior + 12 pairing unit tests including the Python canonical-bytes golden cross-check) - `cargo check --target wasm32-unknown-unknown` on frontend: clean, 0 errors - `bash -n tests/pairing_integration.sh`: syntax OK **Remote integration against `dev-app.projectmycelium.org`:** - `tests/pairing_integration.sh`: **11/11 passed, 0 failed** - Test 1: Farmer SPA keypair registration → 200 - Test 2: Valid signed pair → 201 - Test 3: GET pending list contains the paired node → count=1 - Test 4: Publish endpoint reachable (returns 502 from hero_ledger — see "out of scope" below; state correctly preserved on gateway failure) - Test 5: Idempotent replay → 200 with `idempotent_replay=true` - Test 6: Tampered base64 signature → 401 - Test 7: Stale timestamp (10 min old) → 401 - Test 8: Named form `farmer_account="fred.dev.hero"` → 400 with "implicit accounts only" error **Regression (focused — no touches to these layers but verifying blast radius is zero):** - `api_smoke.sh` → **27/27 passed** - `provider_integration.sh` → **34/34 passed** - `farmer_integration.sh` → **34/34 passed** Zero regressions on the three most-at-risk suites. Full `make test-all` sweep not re-run because the Phase 5 changes are additive (new module, new controller file, three new routes, one new frontend component) with no edits to existing handlers. ### Out of scope for Phase 5 (follow-up) **The publish endpoint's 502 is not a Phase 5 bug.** Dev VM backend logs show: ``` [nodes::publish] ledger listing creation failed for node 01cy: RPC error (-32602): Invalid params: V3 listings must use marketplace.createV3Listing with TF Chain ownership proof ``` The hero_ledger marketplace contract has evolved since Phase 1 landed — it now requires `marketplace.createV3Listing` with on-chain ownership proof, and rejects the legacy `marketplace.create_listing` call shape that both Phase 1's `add_node_from_explorer` **and** my Phase 5 `publish` handler build. Both handlers need to migrate to the v3 method in a follow-up session. Phase 5's own logic correctly: signature-verifies, creates the FarmNode in pending state, filters on pending, calls the (now-wrong) method, returns 502 without state mutation on gateway failure — exactly the specified contract. ### What's deferred (explicit) 1. **F24 "Auto-publish toggle"** — Requires a user preference field + conditional publish-on-pair path in the pair handler. Cut from this session to keep Phase 5 focused on the review-then-approve UX that matches vision.md §3 steps 11-13. Follow-up session. 2. **`HOSTING_VERIFICATION=real`** — Handler stub returns `501 Not Implemented` with a clear message pointing at the two unblock steps: - hero_ledger#45 merge → `hosting.dev.hero` deploy on devnet - hero_ledger gateway client gets `hosting.rs` methods (Phase 2.5 parallel workstream, no upstream dep) Flipping to real is a ~20 LOC diff in `nodes::pair` once those are ready. 3. **`PricingTemplate`** — Phase 6. The publish endpoint currently accepts `{hourly_spore: u64}`; the swap to `{pricing_template_id}` is a one-line body field change when the OSIS root object lands. 4. **Deletion of Phase 1 `/api/dashboard/nodes/from-explorer`** — Retained as a fallback until Phase 3.5 ships the node-side caller. Delete in the same PR that lands Phase 3.5. 5. **Playwright E2E coverage for the new UI** — API is covered by `pairing_integration.sh`; a browser regression belongs in the same session that lands Phase 3.5 real-node pairing, when there'll be a deterministic fixture path to drive it. ### Phase status after this session | Phase | Status | Owner | |---|---|---| | 1 — Crude bridge | ✅ Live on dev VM (retained as fallback) | mik-tf | | 2 — Gateway hosting methods | 🔄 Deferred to Phase 2.5 parallel workstream | — | | 3 — Node crypto identity (hero_compute) | 🟡 Code complete on [#91](https://forge.ourworld.tf/lhumina_code/hero_compute/pulls/91), awaiting **@mahmoud** review | @mahmoud | | 3.5 — First-boot registration hook | ⏳ Blocked on [hero_ledger#45](https://forge.ourworld.tf/lhumina_code/hero_ledger/pulls/45) merge + devnet hosting deploy | @scott (deploy), @mahmoud (hook) | | 4 — `mkt-prepare-usb` | ⏳ Ready to start, no upstream dep | mik-tf | | **5 — Auto-pairing + pending UI (marketplace side)** | ✅ **Shipped 2026-04-11 on `development_mik02`** | mik-tf | | 6 — Pricing templates | Stretch — one-line swap in publish endpoint when templates land | — | | 7 — FarmNode → pure cache | Stretch — requires 3.5 live + 5 merged | — | ### Upstream PR status (unchanged this session — we do not self-merge upstream) - [hero_compute#91](https://forge.ourworld.tf/lhumina_code/hero_compute/pulls/91) — Phase 3 Steps 1-4, CI green, awaiting **@mahmoud** review - [hero_ledger#45](https://forge.ourworld.tf/lhumina_code/hero_ledger/pulls/45) — CI blocked by runner-side git auth bug (not our diff), awaiting **@scott** Per the upstream-PR policy: marketplace repos ship fast via direct push to `development_mik02`, upstream `hero_*` and `mos_*` repos always go via PRs and the maintainer merges. Signed-off-by: mik-tf
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
meeting-notes

Meeting notes and follow-ups

  
meeting-sensitive

Sensitive meeting; no shared archive or AI processing

  
meeting-transcript

Raw meeting transcript; AI drafts the report

No labels
meeting-notes
meeting-sensitive
meeting-transcript
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
coopcloud_code/home#72
No description provided.