Zinit enters runaway process-killing loop, leading to system crash #10

Open
opened 2026-01-27 14:46:11 +00:00 by salmaelsoly · 1 comment
Member

To reproduce:

  • run zinit-server
  • run embedderd --start
  • wait after embedderd starts, then stop it and remove it
  • run embedderd --start but without process_filter service config
  • stop zinit and rerun it
2026-01-27T14:24:36.552885Z  INFO killing process tree parent_pid=3166 children_count=2 pids=[3189, 3188, 3166]
2026-01-27T14:24:36.653089Z  INFO process tree killed successfully parent_pid=3166 killed_count=3
2026-01-27T14:24:36.716594Z  INFO killing process tree parent_pid=39388 children_count=0 pids=[39388]
2026-01-27T14:24:36.816758Z  INFO process tree killed successfully parent_pid=39388 killed_count=1
2026-01-27T14:24:36.874682Z  INFO killing process tree parent_pid=39295 children_count=3 pids=[39304, 39300, 39299, 39295]
2026-01-27T14:24:36.974814Z  INFO process tree killed successfully parent_pid=39295 killed_count=4
2026-01-27T14:24:37.035814Z  INFO killing process tree parent_pid=40980 children_count=20 pids=[43787, 43686, 41320, 41206, 41098, 41089, 41069, 41068, 41067, 41066, 41065, 41064, 41063, 41062, 41040, 41032, 41030, 41029, 41028, 41014, 40980]
2026-01-27T14:24:37.136034Z ERROR CRITICAL: process still alive after SIGKILL pid=40980
2026-01-27T14:24:37.136078Z ERROR CRITICAL: some processes survived SIGKILL parent_pid=40980 killed_count=20 failed_count=1 failed_pids=[40980]
2026-01-27T14:24:37.198347Z  INFO killing process tree parent_pid=3449 children_count=0 pids=[3449]
2026-01-27T14:24:37.198399Z  WARN failed to send SIGKILL to parent process pid=3449 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted
2026-01-27T14:24:37.298506Z  INFO process tree killed successfully parent_pid=3449 killed_count=1
2026-01-27T14:24:37.360453Z  INFO killing process tree parent_pid=41306 children_count=0 pids=[41306]
2026-01-27T14:24:37.460703Z  INFO process tree killed successfully parent_pid=41306 killed_count=1
2026-01-27T14:24:37.521069Z  INFO killing process tree parent_pid=41866 children_count=0 pids=[41866]
2026-01-27T14:24:37.621243Z  INFO process tree killed successfully parent_pid=41866 killed_count=1
2026-01-27T14:24:37.690430Z  INFO killing process tree parent_pid=44124 children_count=0 pids=[44124]
2026-01-27T14:24:37.790629Z  INFO process tree killed successfully parent_pid=44124 killed_count=1
2026-01-27T14:24:37.857669Z  INFO killing process tree parent_pid=43528 children_count=0 pids=[43528]
2026-01-27T14:24:37.957884Z  INFO process tree killed successfully parent_pid=43528 killed_count=1
2026-01-27T14:24:38.018938Z  INFO killing process tree parent_pid=44128 children_count=0 pids=[44128]
2026-01-27T14:24:38.018980Z  WARN failed to send SIGKILL to parent process pid=44128 error=signal error for service '' (signal SIGKILL): ESRCH: No such process
2026-01-27T14:24:38.119090Z  INFO process tree killed successfully parent_pid=44128 killed_count=1
2026-01-27T14:24:38.178167Z  INFO killing process tree parent_pid=41343 children_count=0 pids=[41343]
2026-01-27T14:24:38.278382Z  INFO process tree killed successfully parent_pid=41343 killed_count=1
2026-01-27T14:24:38.330310Z  INFO killing process tree parent_pid=833 children_count=0 pids=[833]
2026-01-27T14:24:38.330353Z  WARN failed to send SIGKILL to parent process pid=833 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted
^X2026-01-27T14:24:38.430506Z  INFO process tree killed successfully parent_pid=833 killed_count=1
2026-01-27T14:24:38.479079Z  INFO killing process tree parent_pid=3441 children_count=0 pids=[3441]
2026-01-27T14:24:38.479119Z  WARN failed to send SIGKILL to parent process pid=3441 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted
^C2026-01-27T14:24:38.508629Z  INFO Ctrl+C received, initiating shutdown
2026-01-27T14:24:38.508724Z  INFO shutdown requested via IPC
2026-01-27T14:24:38.579325Z  INFO process tree killed successfully parent_pid=3441 killed_count=1
2026-01-27T14:24:38.628194Z  INFO killing process tree parent_pid=44160 children_count=0 pids=[44160]
^C2026-01-27T14:24:38.728493Z  INFO process tree killed successfully parent_pid=44160 killed_count=1
^C2026-01-27T14:24:38.777640Z  INFO killing process tree parent_pid=44007 children_count=0 pids=[44007]
2026-01-27T14:24:38.777680Z  WARN failed to send SIGKILL to parent process pid=44007 error=signal error for service '' (signal SIGKILL): ESRCH: No such process
^C2026-01-27T14:24:38.877889Z  INFO process tree killed successfully parent_pid=44007 killed_count=1
2026-01-27T14:24:38.925158Z  INFO killing process tree parent_pid=2744 children_count=0 pids=[2744]
2026-01-27T14:24:38.925193Z  WARN failed to send SIGKILL to parent process pid=2744 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted
^C2026-01-27T14:24:39.025433Z  INFO process tree killed successfully parent_pid=2744 killed_count=1
2026-01-27T14:24:39.074150Z  INFO killing process tree parent_pid=41201 children_count=0 pids=[41201]
2026-01-27T14:24:39.174381Z  INFO process tree killed successfully parent_pid=41201 killed_count=1
2026-01-27T14:24:39.220778Z  INFO killing process tree parent_pid=39675 children_count=0 pids=[39675]
2026-01-27T14:24:39.320954Z  INFO process tree killed successfully parent_pid=39675 killed_count=1
2026-01-27T14:24:39.366540Z  INFO killing process tree parent_pid=41273 children_count=0 pids=[41273]
2026-01-27T14:24:39.466718Z  INFO process tree killed successfully parent_pid=41273 killed_count=1
2026-01-27T14:24:39.512344Z  INFO killing process tree parent_pid=1184 children_count=0 pids=[1184]
2026-01-27T14:24:39.512390Z  WARN failed to send SIGKILL to parent process pid=1184 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted
^C2026-01-27T14:24:39.612651Z  INFO process tree killed successfully parent_pid=1184 killed_count=1
2026-01-27T14:24:39.657365Z  INFO killing process tree parent_pid=39383 children_count=0 pids=[39383]
^C2026-01-27T14:24:39.757607Z  INFO process tree killed successfully parent_pid=39383 killed_count=1
2026-01-27T14:24:39.802641Z  INFO killing process tree parent_pid=41305 children_count=0 pids=[41305]
2026-01-27T14:24:39.902933Z  INFO process tree killed successfully parent_pid=41305 killed_count=1
2026-01-27T14:24:39.947124Z  INFO killing process tree parent_pid=184 children_count=0 pids=[184]
2026-01-27T14:24:39.947160Z  WARN failed to send SIGKILL to parent process pid=184 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted
2026-01-27T14:24:40.047278Z  INFO process tree killed successfully parent_pid=184 killed_count=1
2026-01-27T14:24:40.092761Z  INFO killing process tree parent_pid=43689 children_count=0 pids=[43689]
2026-01-27T14:24:40.192928Z  INFO process tree killed successfully parent_pid=43689 killed_count=1
2026-01-27T14:24:40.236098Z  INFO killing process tree parent_pid=44355 children_count=0 pids=[44355]
2026-01-27T14:24:40.336289Z  INFO process tree killed successfully parent_pid=44355 killed_count=1
2026-01-27T14:24:40.379772Z  INFO killing process tree parent_pid=1470 children_count=0 pids=[1470]
2026-01-27T14:24:40.379808Z  WARN failed to send SIGKILL to parent process pid=1470 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted
2026-01-27T14:24:40.479953Z  INFO process tree killed successfully parent_pid=1470 killed_count=1
2026-01-27T14:24:40.527312Z  INFO killing process tree parent_pid=41492 children_count=0 pids=[41492]
2026-01-27T14:24:40.527354Z  WARN failed to send SIGKILL to parent process pid=41492 error=signal error for service '' (signal SIGKILL): ESRCH: No such process
2026-01-27T14:24:40.627459Z  INFO process tree killed successfully parent_pid=41492 killed_count=1
2026-01-27T14:24:40.667288Z  INFO killing process tree parent_pid=2577 children_count=0 pids=[2577]
2026-01-27T14:24:40.667327Z  WARN failed to send SIGKILL to parent process pid=2577 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted
2026-01-27T14:24:40.767460Z  INFO process tree killed successfully parent_pid=2577 killed_count=1
2026-01-27T14:24:40.807893Z  INFO killing process tree parent_pid=2748 children_count=0 pids=[2748]
2026-01-27T14:24:40.807933Z  WARN failed to send SIGKILL to parent process pid=2748 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted
2026-01-27T14:24:40.908051Z  INFO process tree killed successfully parent_pid=2748 killed_count=1
^C2026-01-27T14:24:40.952010Z  INFO killing process tree parent_pid=39393 children_count=0 pids=[39393]
2026-01-27T14:24:41.052184Z  INFO process tree killed successfully parent_pid=39393 killed_count=1
^C2026-01-27T14:24:41.094792Z  INFO killing process tree parent_pid=39497 children_count=0 pids=[39497]
2026-01-27T14:24:41.194955Z  INFO process tree killed successfully parent_pid=39497 killed_count=1
^C2026-01-27T14:24:41.238093Z  INFO killing process tree parent_pid=1348 children_count=0 pids=[1348]
2026-01-27T14:24:41.238140Z  WARN failed to send SIGKILL to parent process pid=1348 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted
2026-01-27T14:24:41.338234Z  INFO process tree killed successfully parent_pid=1348 killed_count=1
^C2026-01-27T14:24:41.379238Z  INFO killing process tree parent_pid=40998 children_count=0 pids=[40998]
2026-01-27T14:24:41.479391Z  INFO process tree killed successfully parent_pid=40998 killed_count=1
2026-01-27T14:24:41.506592Z  INFO killing process tree parent_pid=41954 children_count=0 pids=[41954]
2026-01-27T14:24:41.506637Z  WARN failed to send SIGKILL to parent process pid=41954 error=signal error for service '' (signal SIGKILL): ESRCH: No such process
2026-01-27T14:24:41.606745Z  INFO process tree killed successfully parent_pid=41954 killed_count=1
2026-01-27T14:24:41.635619Z  INFO killing process tree parent_pid=44062 children_count=0 pids=[44062]

### To reproduce: - run zinit-server - run embedderd --start - wait after embedderd starts, then stop it and remove it - run embedderd --start but without process_filter service config - stop zinit and rerun it ``` 2026-01-27T14:24:36.552885Z INFO killing process tree parent_pid=3166 children_count=2 pids=[3189, 3188, 3166] 2026-01-27T14:24:36.653089Z INFO process tree killed successfully parent_pid=3166 killed_count=3 2026-01-27T14:24:36.716594Z INFO killing process tree parent_pid=39388 children_count=0 pids=[39388] 2026-01-27T14:24:36.816758Z INFO process tree killed successfully parent_pid=39388 killed_count=1 2026-01-27T14:24:36.874682Z INFO killing process tree parent_pid=39295 children_count=3 pids=[39304, 39300, 39299, 39295] 2026-01-27T14:24:36.974814Z INFO process tree killed successfully parent_pid=39295 killed_count=4 2026-01-27T14:24:37.035814Z INFO killing process tree parent_pid=40980 children_count=20 pids=[43787, 43686, 41320, 41206, 41098, 41089, 41069, 41068, 41067, 41066, 41065, 41064, 41063, 41062, 41040, 41032, 41030, 41029, 41028, 41014, 40980] 2026-01-27T14:24:37.136034Z ERROR CRITICAL: process still alive after SIGKILL pid=40980 2026-01-27T14:24:37.136078Z ERROR CRITICAL: some processes survived SIGKILL parent_pid=40980 killed_count=20 failed_count=1 failed_pids=[40980] 2026-01-27T14:24:37.198347Z INFO killing process tree parent_pid=3449 children_count=0 pids=[3449] 2026-01-27T14:24:37.198399Z WARN failed to send SIGKILL to parent process pid=3449 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted 2026-01-27T14:24:37.298506Z INFO process tree killed successfully parent_pid=3449 killed_count=1 2026-01-27T14:24:37.360453Z INFO killing process tree parent_pid=41306 children_count=0 pids=[41306] 2026-01-27T14:24:37.460703Z INFO process tree killed successfully parent_pid=41306 killed_count=1 2026-01-27T14:24:37.521069Z INFO killing process tree parent_pid=41866 children_count=0 pids=[41866] 2026-01-27T14:24:37.621243Z INFO process tree killed successfully parent_pid=41866 killed_count=1 2026-01-27T14:24:37.690430Z INFO killing process tree parent_pid=44124 children_count=0 pids=[44124] 2026-01-27T14:24:37.790629Z INFO process tree killed successfully parent_pid=44124 killed_count=1 2026-01-27T14:24:37.857669Z INFO killing process tree parent_pid=43528 children_count=0 pids=[43528] 2026-01-27T14:24:37.957884Z INFO process tree killed successfully parent_pid=43528 killed_count=1 2026-01-27T14:24:38.018938Z INFO killing process tree parent_pid=44128 children_count=0 pids=[44128] 2026-01-27T14:24:38.018980Z WARN failed to send SIGKILL to parent process pid=44128 error=signal error for service '' (signal SIGKILL): ESRCH: No such process 2026-01-27T14:24:38.119090Z INFO process tree killed successfully parent_pid=44128 killed_count=1 2026-01-27T14:24:38.178167Z INFO killing process tree parent_pid=41343 children_count=0 pids=[41343] 2026-01-27T14:24:38.278382Z INFO process tree killed successfully parent_pid=41343 killed_count=1 2026-01-27T14:24:38.330310Z INFO killing process tree parent_pid=833 children_count=0 pids=[833] 2026-01-27T14:24:38.330353Z WARN failed to send SIGKILL to parent process pid=833 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted ^X2026-01-27T14:24:38.430506Z INFO process tree killed successfully parent_pid=833 killed_count=1 2026-01-27T14:24:38.479079Z INFO killing process tree parent_pid=3441 children_count=0 pids=[3441] 2026-01-27T14:24:38.479119Z WARN failed to send SIGKILL to parent process pid=3441 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted ^C2026-01-27T14:24:38.508629Z INFO Ctrl+C received, initiating shutdown 2026-01-27T14:24:38.508724Z INFO shutdown requested via IPC 2026-01-27T14:24:38.579325Z INFO process tree killed successfully parent_pid=3441 killed_count=1 2026-01-27T14:24:38.628194Z INFO killing process tree parent_pid=44160 children_count=0 pids=[44160] ^C2026-01-27T14:24:38.728493Z INFO process tree killed successfully parent_pid=44160 killed_count=1 ^C2026-01-27T14:24:38.777640Z INFO killing process tree parent_pid=44007 children_count=0 pids=[44007] 2026-01-27T14:24:38.777680Z WARN failed to send SIGKILL to parent process pid=44007 error=signal error for service '' (signal SIGKILL): ESRCH: No such process ^C2026-01-27T14:24:38.877889Z INFO process tree killed successfully parent_pid=44007 killed_count=1 2026-01-27T14:24:38.925158Z INFO killing process tree parent_pid=2744 children_count=0 pids=[2744] 2026-01-27T14:24:38.925193Z WARN failed to send SIGKILL to parent process pid=2744 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted ^C2026-01-27T14:24:39.025433Z INFO process tree killed successfully parent_pid=2744 killed_count=1 2026-01-27T14:24:39.074150Z INFO killing process tree parent_pid=41201 children_count=0 pids=[41201] 2026-01-27T14:24:39.174381Z INFO process tree killed successfully parent_pid=41201 killed_count=1 2026-01-27T14:24:39.220778Z INFO killing process tree parent_pid=39675 children_count=0 pids=[39675] 2026-01-27T14:24:39.320954Z INFO process tree killed successfully parent_pid=39675 killed_count=1 2026-01-27T14:24:39.366540Z INFO killing process tree parent_pid=41273 children_count=0 pids=[41273] 2026-01-27T14:24:39.466718Z INFO process tree killed successfully parent_pid=41273 killed_count=1 2026-01-27T14:24:39.512344Z INFO killing process tree parent_pid=1184 children_count=0 pids=[1184] 2026-01-27T14:24:39.512390Z WARN failed to send SIGKILL to parent process pid=1184 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted ^C2026-01-27T14:24:39.612651Z INFO process tree killed successfully parent_pid=1184 killed_count=1 2026-01-27T14:24:39.657365Z INFO killing process tree parent_pid=39383 children_count=0 pids=[39383] ^C2026-01-27T14:24:39.757607Z INFO process tree killed successfully parent_pid=39383 killed_count=1 2026-01-27T14:24:39.802641Z INFO killing process tree parent_pid=41305 children_count=0 pids=[41305] 2026-01-27T14:24:39.902933Z INFO process tree killed successfully parent_pid=41305 killed_count=1 2026-01-27T14:24:39.947124Z INFO killing process tree parent_pid=184 children_count=0 pids=[184] 2026-01-27T14:24:39.947160Z WARN failed to send SIGKILL to parent process pid=184 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted 2026-01-27T14:24:40.047278Z INFO process tree killed successfully parent_pid=184 killed_count=1 2026-01-27T14:24:40.092761Z INFO killing process tree parent_pid=43689 children_count=0 pids=[43689] 2026-01-27T14:24:40.192928Z INFO process tree killed successfully parent_pid=43689 killed_count=1 2026-01-27T14:24:40.236098Z INFO killing process tree parent_pid=44355 children_count=0 pids=[44355] 2026-01-27T14:24:40.336289Z INFO process tree killed successfully parent_pid=44355 killed_count=1 2026-01-27T14:24:40.379772Z INFO killing process tree parent_pid=1470 children_count=0 pids=[1470] 2026-01-27T14:24:40.379808Z WARN failed to send SIGKILL to parent process pid=1470 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted 2026-01-27T14:24:40.479953Z INFO process tree killed successfully parent_pid=1470 killed_count=1 2026-01-27T14:24:40.527312Z INFO killing process tree parent_pid=41492 children_count=0 pids=[41492] 2026-01-27T14:24:40.527354Z WARN failed to send SIGKILL to parent process pid=41492 error=signal error for service '' (signal SIGKILL): ESRCH: No such process 2026-01-27T14:24:40.627459Z INFO process tree killed successfully parent_pid=41492 killed_count=1 2026-01-27T14:24:40.667288Z INFO killing process tree parent_pid=2577 children_count=0 pids=[2577] 2026-01-27T14:24:40.667327Z WARN failed to send SIGKILL to parent process pid=2577 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted 2026-01-27T14:24:40.767460Z INFO process tree killed successfully parent_pid=2577 killed_count=1 2026-01-27T14:24:40.807893Z INFO killing process tree parent_pid=2748 children_count=0 pids=[2748] 2026-01-27T14:24:40.807933Z WARN failed to send SIGKILL to parent process pid=2748 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted 2026-01-27T14:24:40.908051Z INFO process tree killed successfully parent_pid=2748 killed_count=1 ^C2026-01-27T14:24:40.952010Z INFO killing process tree parent_pid=39393 children_count=0 pids=[39393] 2026-01-27T14:24:41.052184Z INFO process tree killed successfully parent_pid=39393 killed_count=1 ^C2026-01-27T14:24:41.094792Z INFO killing process tree parent_pid=39497 children_count=0 pids=[39497] 2026-01-27T14:24:41.194955Z INFO process tree killed successfully parent_pid=39497 killed_count=1 ^C2026-01-27T14:24:41.238093Z INFO killing process tree parent_pid=1348 children_count=0 pids=[1348] 2026-01-27T14:24:41.238140Z WARN failed to send SIGKILL to parent process pid=1348 error=signal error for service '' (signal SIGKILL): EPERM: Operation not permitted 2026-01-27T14:24:41.338234Z INFO process tree killed successfully parent_pid=1348 killed_count=1 ^C2026-01-27T14:24:41.379238Z INFO killing process tree parent_pid=40998 children_count=0 pids=[40998] 2026-01-27T14:24:41.479391Z INFO process tree killed successfully parent_pid=40998 killed_count=1 2026-01-27T14:24:41.506592Z INFO killing process tree parent_pid=41954 children_count=0 pids=[41954] 2026-01-27T14:24:41.506637Z WARN failed to send SIGKILL to parent process pid=41954 error=signal error for service '' (signal SIGKILL): ESRCH: No such process 2026-01-27T14:24:41.606745Z INFO process tree killed successfully parent_pid=41954 killed_count=1 2026-01-27T14:24:41.635619Z INFO killing process tree parent_pid=44062 children_count=0 pids=[44062] ```
despiegk added this to the now milestone 2026-02-05 07:04:27 +00:00
Owner

Can someone update the status here? Is it fixed? not yet?

Can someone update the status here? Is it fixed? not yet?
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
geomind_code/zinit#10
No description provided.