v0.3.2: spawn_blocking fix, static builds, CI overhaul #3

Merged

mik-tf merged 24 commits from development into main 2026-02-06 14:29:31 +00:00

Conversation 1 Commits 24 Files changed 22 +3367 -1967

mik-tf commented 2026-02-06 04:12:42 +00:00

Owner

Copy link

Summary

Brings development up to main with all changes since v0.0.1. Key areas: runtime fix for AUTH/SAUTH hangs, static binary builds, unified build system, and CI fixes.

Changes

Runtime fix: spawn_blocking for redb I/O

All async methods in backend.rs that performed blocking redb disk I/O while holding tokio RwLock guards have been wrapped in tokio::task::spawn_blocking(). This fixes AUTH/SAUTH hangs caused by blocking the tokio runtime thread.

Static binary builds

• Added mimalloc as global allocator for musl targets
• Added rust-toolchain.toml and cargo config for musl builds
• Static binary builds via build-linux.yaml and build-macos.yaml workflows
• Tag-based versioned publishing to Forge package registry (v0.1.0 through v0.1.3)

Build system

• buildenv.sh: single source of truth for project configuration
• scripts/build_lib.sh: shared build functions (cargo env, install, publish, release)
• Makefile: unified targets (build, test, install, ship, release)
• VERSION file for version tracking

CI

• build.yaml: test workflow on every push (fixed cargo env sourcing)
• build-linux.yaml: release builds on tag push (x86_64 musl + aarch64 cross-compile)
• build-macos.yaml: macOS release builds on tag push

Version

Unified version to 0.3.2 across Cargo.toml, VERSION, and buildenv.sh.

Testing

• 14 unit tests pass
• 60 integration tests pass (hero_redis_tester)
• CI runs #76 and #77 both pass
• hero_books builds and tests pass against these changes (client API unchanged)

## Summary Brings development up to main with all changes since v0.0.1. Key areas: runtime fix for AUTH/SAUTH hangs, static binary builds, unified build system, and CI fixes. ## Changes ### Runtime fix: spawn_blocking for redb I/O All async methods in `backend.rs` that performed blocking redb disk I/O while holding tokio RwLock guards have been wrapped in `tokio::task::spawn_blocking()`. This fixes AUTH/SAUTH hangs caused by blocking the tokio runtime thread. ### Static binary builds - Added `mimalloc` as global allocator for musl targets - Added `rust-toolchain.toml` and cargo config for musl builds - Static binary builds via `build-linux.yaml` and `build-macos.yaml` workflows - Tag-based versioned publishing to Forge package registry (v0.1.0 through v0.1.3) ### Build system - `buildenv.sh`: single source of truth for project configuration - `scripts/build_lib.sh`: shared build functions (cargo env, install, publish, release) - `Makefile`: unified targets (build, test, install, ship, release) - `VERSION` file for version tracking ### CI - `build.yaml`: test workflow on every push (fixed cargo env sourcing) - `build-linux.yaml`: release builds on tag push (x86_64 musl + aarch64 cross-compile) - `build-macos.yaml`: macOS release builds on tag push ### Version Unified version to `0.3.2` across `Cargo.toml`, `VERSION`, and `buildenv.sh`. ## Testing - 14 unit tests pass - 60 integration tests pass (hero_redis_tester) - CI runs #76 and #77 both pass - hero_books builds and tests pass against these changes (client API unchanged)

mik-tf added 3 commits 2026-02-06 04:12:42 +00:00

fix(backend): wrap blocking redb operations in spawn_blocking ... 940839670f

The AUTH/SAUTH commands were hanging because blocking redb I/O operations
(begin_read, open_table, table.get) were being called while holding tokio
async RwLock guards. This blocked the tokio worker threads and caused
deadlocks.

Fixed by wrapping the blocking operations in tokio::task::spawn_blocking()
for the critical methods used in the auth path:
- hget(): Used to look up admin secrets
- get(): Used to retrieve user permissions
- keys(): Used to scan for database ACLs

The spawn_blocking() moves the blocking I/O to a dedicated thread pool,
preventing it from blocking the async executor.

This is the industry-standard pattern for using synchronous database
libraries (like redb) in async Rust code.

Tested: AUTH and SAUTH commands now work correctly without hanging.

fix: resolve AUTH/SAUTH hang by using spawn_blocking for redb I/O ... 3ab187477d

- Convert all async methods in backend.rs to use tokio::task::spawn_blocking()
- Use blocking_read()/blocking_write() instead of read().await/write().await
- Prevents deadlocks from blocking redb I/O while holding tokio RwLock guards
- Fixed ~60+ methods across TTL, String, Hash, List, Set, Stream, and Utility operations
- All tests pass

style: fix formatting for CI 866900adb1

mik-tf commented 2026-02-06 04:16:31 +00:00

Author

Owner

Copy link

@timur @despiegk

@timur @despiegk

despiegk added 16 commits 2026-02-06 04:57:34 +00:00

chore: add mimalloc as global allocator for musl targets f1ca4a07da

refactor: reorganize ports to 33xx (backend) / 88xx (frontend) scheme ... e5f95a3058

Port mapping:
- hero_zero:     3377 (unchanged)
- hero_redis:    6666 → 3378
- hero_db:       5555 → 3379
- hero_embedder: 3752 → 3380
- hero_voice:    2756 → 3381
- hero_os:       6438 → 8880
- hero_biz:      3388 → 8881
- hero_editor:   2769 → 8882
- hero_books:    9567 → 8883
- hero_browser:  4829 → 8884
- hero_forge_ui: 8654 → 8885
- zinit:         3875 (unchanged, not Hero suite)

E2E: 293/295 pass (2 pre-existing Whisper model failures)

chore: add rust-toolchain.toml and cargo config for musl builds d831e778f7

chore: add Makefile with build, test, install, and dev targets b00f2e2771

ci: build fully static binaries (musl/crt-static) a0b83691c2

docs: standardize README with Install from Binaries and Build from Source sections ... f652df9cfe

- Add Forge package registry link for pre-built binary download
- Add curl command for direct binary download to ~/hero/bin
- Standardize Build from Source to use make build
- Link to specific package page on forge.ourworld.tf

makefile: add ship-binary target (tag + push to trigger CI) eef4f6d2a0

ci: publish packages under tag version instead of hardcoded dev bb36c68966

ci: trigger v0.1.2 build 9317bb3941

chore: add VERSION file (0.1.3), sync Cargo.toml, Makefile reads from VERSION e135f5e3c9

ci: trigger v0.1.3 build 7a29fa8003

merge: combine development changes with comprehensive Makefile targets ... 8b10d56738

Merge development branch into development-static-build, combining:
- VERSION file-based versioning (from HEAD)
- Comprehensive build and deployment targets (from HEAD)
- Ship-binary CI trigger (from HEAD)
- Release management targets: release-patch, release-minor, release-major (from development)

All targets and features are now available in a unified Makefile.

chore: simplify build targets - keep only single build target ... 34c63ca603

Remove redundant build-server and build-client targets. Single 'build' target now handles building release binaries.

chore: simplify build targets - keep only single build target ... 28103c7425

Remove redundant build-server and build-client targets. Single 'build' target now handles building release binaries.

Merge branch 'development-static-build' of https://forge.ourworld.tf/geomind_code/hero_redis into development-static-build ... 06a854dc6f

* 'development-static-build' of https://forge.ourworld.tf/geomind_code/hero_redis:
  chore: simplify build targets - keep only single build target

Merge branch 'development' into development-static-build ... 4a11e59c42

* development:
  style: fix formatting for CI
  fix: resolve AUTH/SAUTH hang by using spawn_blocking for redb I/O
  fix(backend): wrap blocking redb operations in spawn_blocking

despiegk added 2 commits 2026-02-06 06:05:15 +00:00

better buildscripts f2e99371f6

build scripts updates 43ac212c59

despiegk added 1 commit 2026-02-06 06:26:17 +00:00

lib better 6a80ffd05a

mik-tf added 1 commit 2026-02-06 14:08:00 +00:00

fix: CI cargo env sourcing, unify version to 0.3.2 ... 286594323e

- build.yaml: add cargo env sourcing before make commands (fixes 'cargo: No such file or directory')
- Unify version across Cargo.toml, VERSION, and buildenv.sh to 0.3.2
- Regenerate Cargo.lock

mik-tf changed title from fix: resolve AUTH/SAUTH hang by using spawn_blocking for redb I/O to v0.3.2: spawn_blocking fix, static builds, CI overhaul 2026-02-06 14:22:10 +00:00

mik-tf added 1 commit 2026-02-06 14:25:50 +00:00

merge: sync main into development before PR merge aebf275064

mik-tf merged commit 4096327f0c into main 2026-02-06 14:29:31 +00:00

mik-tf referenced this pull request from a commit 2026-02-06 14:29:33 +00:00

Merge pull request 'v0.3.2: spawn_blocking fix, static builds, CI overhaul' (#3) from development into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

lhumina_code/hero_redis!3

No description provided.