GitHub

User Guide

Command reference. For a hands-on path, use First Steps (Requirements → Prepare → Demo).

Full flag lists: julia --project=. -m DistSSHKit {cmd} --help. Each command page starts with a Flags table for that command.

setupCheck hosts, clone / rsync / sync, instantiate, cleanup
goStandalone script as-is; one full run per slot
driveMaster + Distributed workers; driver farms work
sizeEstimate worker counts from RAM / CPU
demoInstall or list bundled example scripts
distsshkitOptional terminal command (pkg> app add; experimental)

go vs drive (pick one)

Both share host tokens (local:N, host:N) and optional --sync / --rsync. The difference is what the script is:

godrive
ScriptOrdinary .jl (no Kit APIs)Driver with init_output_dir! / main
host:N meansN concurrent full script runsN Distributed workers
CollectSlot-overwrite after remotesPost-run-new after main(); optional collect-only flags
Git parityNo --require-gitOpt-in --require-git
--output-dirBatch root (PATH/{slot}/)Result root (DISTRIBUTED_OUTPUT_DIR)

If you want “run this job on a few machines,” start with go. If you want one master farming work with pmap (and friends), use drive.

Flag consistency (read once)

Same names are shared on purpose; a few meanings differ by command:

TopicRule
--sync / --rsyncSame git vs rsync idea on setup (mode) and go / drive (optional pre-run). On each command, pick at most one.
Default pre-run syncgo and drive: none (run setup yourself, or pass --sync / --rsync).
Git parity (drive)Off by default. Opt-in: --require-git. Compat: --skip-git-guard (no-op; may combine with --sync / --rsync).
Skip pre-run (go)Compat: --skip-sync / --skip-git-guard (already the default; exclusive with --sync / --rsync on go).
--output-dirgo: batch root (PATH/{slot}/). drive: result root (DISTRIBUTED_OUTPUT_DIR). Different on purpose.
-l / --localdrive: local:N worker count. size: include localhost (boolean).
--hostsCSV tokens. setup / size strip :N. go / drive keep host:N.
--hosts-filesetup / size strip :N. go / drive keep host:N for slots / workers.
Shared peel-q/--quiet, --progress, --verbose, -y/--yes, --hosts, --hosts-file, -v/--version — same on setup / go / drive / size.

Shared concepts

Hosts. Sources, in the order they append after positional tokens:

  • CLI tokens (local:N, host:N) on setup / go / drive / size
  • --hosts (CSV)
  • DISTSSHKIT_HOSTS (comma-separated)
  • --hosts-file (default path from DISTSSHKIT_HOSTS_FILE)

setup / size strip :N and use host names only.

Jobs. DISTSSHKIT_JOBS (default 1) is the max concurrent SSH host jobs for setup --rsync, drive post-run collect, and size Julia detection. Worker addprocs stays sequential.

Quiet / progress.

  • Default: --progress (live status) on a TTY; full detail when piped or NO_COLOR
  • -q hides terminal detail; --verbose forces full detail — at most one (DISTSSHKIT_QUIET / DISTSSHKIT_PROGRESS / DISTSSHKIT_VERBOSE)
  • Kit / slot logs still write regardless; fatals stay on the terminal
  • Confirm prompts always print (-y / DISTSSHKIT_YES skips them)

Stale workers.

  • Local drive workers are torn down with rmprocs, not a pattern pkill
  • Before adding SSH workers, drive may pkill -9 -f julia --worker / julia --bind-to on those remotes
  • setup --cleanup runs that same sweep on localhost and remotes (other Distributed jobs on the same login can match)
  • DISTSSHKIT_SKIP_GLOBAL_WORKER_PKILL=1 skips those pkills; rmprocs still runs for the current drive

Kit files. Logs and go batches live under {project}/.distsshkit/. Add that directory to the job project's .gitignore — DistSSHKit's own repo already ignores it, but Pkg.add does not. Otherwise go output can show up as untracked files, including under drive --require-git.

Collect modes:

ModeWhere
slot-overwritego after each remote slot (whole slot dir)
post-run-newdrive after main() (newer than run-start sentinel)
collect-missingdrive --collect-missing / collect!(merge=false)
collect-overwritedrive --collect-overwrite / collect!(merge=true)