Run External Commands

runcmd module implements a high-level interface for running external commands.

Why not vlib os?

The standard V os module already contains many tools for running external commands, but I don't like any of them. So let's overview.

• os.execvp(), os.execve() — cross-platform versions of the C functions of the same name.

• os.execute(), os.execute_opt(), os.execute_or_exit(), os.execute_or_panic(), os.raw_execute() — starts and waits for a command to completed. Under the hood, they perform a dirty hack by calling shell with stream redirection 'exec 2>&1;${cmd}'.

• util.execute_with_timeout() (from os.util) — just an os.execute() wrapper.

• os.system() — also executes command in the shell, but does not redirect streams. This is fine for running commands that take a long time and write something to the terminal; it's convenient in build scripts.

• os.Process just has an ugly interface with a lot of unnecessary methods. Actually, it's not bad; I copied parts of it.

• os.Command calls C.popen() under the hood and is not suitable for anything other than running a command in the shell (again) with stream processing of the mixed stdout and stderr.

The obvious downside of this module is that it only works on Linux and likely other POSIX-compliant operating systems. I'm not interested in working on MS Windows, but anyone interested can submit a PR on GitHub to support the worst operating system.

Usage

Basic usage:

import runcmd

mut cmd := runcmd.new('sh', '-c', 'echo Hello,  World!')
cmd.run()! // Start and wait for process.
// Hello, World!
println(cmd.state) // exit status 0

If you don't want to wait for the child process to complete, call start() instead of run():

mut cmd := runcmd.new('sh', '-c', 'sleep 60')
pid := cmd.start()!
println(pid)

.state value is unavailable in this case because we didn't wait for the process to complete.

If you need to capture standard output and standard error, use the output() and combined_output(). See examples in its description.

To learn more about the runcmd's capabilities and usage, see the examples directory. Examples are very important.

Roadmap

• Basic implementation.
• Contexts support for creating cancelable commands, commands with timeouts, etc.
• Better error handling and more tests...

Send pull requests for additional features/bugfixes.

fn is_present(cmd string) bool

is_present returns true if cmd is present on system. cmd may be a command name or filepath (relative or absolute). The result relies on look_path() output, see its docs for command search details.

fn look_path(cmd string) !string

look_path returns the absolute path to executable file. cmd may be a command name or filepath (relative or absolute). The command is searched in PATH. If the name contains a slash, then the PATH search is not performed, instead the path will be resolved and the file existence will be checked. In both cases the file must have the execute permission bit enabled.

Note: To use executables located in the current working directory use './file' instead of just 'file'. Searching for executable files in the current directory is disabled for security reasons. See https://go.dev/blog/path-security.

fn new(name string, arg ...string) &Command

new creates new command with given command name and arguments.

fn with_context(ctx context.Context, name string, arg ...string) &Command

with_context creates new command with context, command name and arguments.

type CommandCancelFn = fn () !

type ProcessHookFn = fn (mut p Process) !

fn (s ProcessState) pid() int

pid returns the child process identifier. If process is not launched yet -1 wil be returned.

fn (s ProcessState) exited() bool

exited returns true if process is exited.

fn (s ProcessState) exit_code() int

exit_code returns the process exit status code or -1 if process is not exited.

fn (s ProcessState) success() bool

success returns true if process if successfuly exited (0 exit status on POSIX).

fn (s ProcessState) sys() voidptr

sys returns the system-specific process state object. For now its always WaitStatus.

fn (s ProcessState) str() string

str returns the text representation of process state. For non-started process it returns 'unknown' state.

fn (mut f ReadFd) read(mut buf []u8) !int

read reads the buf.len bytes from file descriptor and returns number of bytes read on success. This function implements the io.Reader interface.

fn (mut f ReadFd) slurp() ![]u8

slurp reads all data from file descriptor (until gets io.Eof) and returns result as byte array.

fn (mut f ReadFd) close() !

close closes the underlying file descriptor.

type WaitStatus = u32

WaitStatus stores the result value of wait(2) syscall.

fn (w WaitStatus) exited() bool

exited returns true if process is exited.

fn (w WaitStatus) exit_code() int

exit_code returns the process exit status code or -1 if process is not exited.

fn (w WaitStatus) signaled() bool

signaled returns true if the child process was terminated by a signal.

fn (w WaitStatus) term_signal() int

term_signal returns the number of the signal that caused the child process to terminate.

fn (w WaitStatus) stopped() bool

stopped returns true if the child process was stopped by delivery of a signal.

fn (w WaitStatus) stop_signal() int

stop_signal returns the number of the signal which caused the child to stop.

fn (w WaitStatus) continued() bool

continued returns true if the child process was resumed by delivery of SIGCONT.

fn (w WaitStatus) coredump() bool

coredump returns true if the child produced a core dump. See core(5).

fn (mut f WriteFd) write(buf []u8) !int

write writes the buf.len bytes to the file descriptor and returns number of bytes written on success. This function implements the io.Writer interface.

fn (mut f WriteFd) close() !

close closes the underlying file descriptor.

struct Command {
pub mut:
	// path may be a command name or absolute path to executable.
	// If the specified path is not absolute, it will be obtained
	// using the look_path() function before starting the process.
	path string

	// args holds command line arguments passed to the executable.
	args []string

	// env contains key-value pairs of environment variables that
	// will be passed to the process. If not specified the current
	// os.environ() will be used. If you want to update current
	// environ instead of overriding it, use merge() function from
	// `maps` module: `maps.merge(os.environ(), {'MYENV': 'value'})`
	env map[string]string

	// dir specifies the working directory for the child process.
	// If not specified, the current working directory of parent
	// will be used.
	dir string

	// If true create pipes for standart in/out/err streams and
	// duplicate child streams to created file descriptors.
	// redirect_stdio is required for work with child I/O.
	redirect_stdio bool

	// If the field is filled (any of stdin, stdout, stderr), then
	// after the process is started, a coroutine will be launched
	// in the background, which will copy the corresponding data
	// stream between the child and parent processes.
	//
	// This fields must be set BEFORE calling the start() function.
	//
	// This is useful if we want to feed data to the child process
	// via standard input or read stdout and/or stderr entirely.
	// Since the coroutines copies the stream continuously in a loop,
	// the data can only be processed once it has been completely
	// read. To process data in chunks, do not set these fields, but
	// use the stdin(), stdout() and stderr() functions to obtain
	// the child process's file descriptors after calling start().
	stdin  ?io.Reader
	stdout ?io.Writer
	stderr ?io.Writer

	// ctx holds a command context. It may be used to make a command
	// cancelable or set timeout/deadline for it.
	ctx ?context.Context

	// cancel function is used to terminate the child process. Do
	// not confuse with the context's cancel function.
	//
	// The default command cancel function created in with_context()
	// terminates the command by sending SIGTERM signal to child. You
	// can override it by setting your own command cancel function.
	// If cancel is none the child process won't be terminated even
	// if context is timed out or canceled.
	cancel ?CommandCancelFn

	// pre_exec_hooks will be called before starting the command in
	// the child process. Hooks can be used to modify a child's envi-
	// ronment, for example to perform a chroot.
	pre_exec_hooks []ProcessHookFn

	// process holds the underlying Process once started.
	process ?&Process

	// state holds an information about underlying process.
	// This is set only if process if finished. Call `run()`
	// or `wait()` to get actual state value.
	// This value MUST NOT be changed by API user.
	state ProcessState
mut:
	// stdio holds a file descriptors for I/O processing.
	// There is:
	// * 0 — child process stdin, we must write into it.
	// * 1 — child process stdout, we must read from it.
	// * 2 — child process stderr, we must read from it.
	stdio [3]int = [-1, -1, -1]!

	// stdio_copy_fns is array of closures to copy data between
	// parent and child processes. For standard I/O streams
	// it does (if some of .stdin, .stdout and .stderr fields is set):
	// * read from .stdin reader and write data into child stdin fd.
	// * read from child stdout fd and write into .stdout writer.
	// * read from child stderr fd and write into .stderr writer.
	stdio_copy_fns []IOCopyFn
}

fn (mut c Command) run() !

run starts a specified command and waits for it. After call see the .state value to get finished process identifier, exit status and other attributes. run() is shorthand for:

cmd.start()!
cmd.wait()!

fn (mut c Command) output() !string

output runs the command and returns its stdout on success. If command exit status is non-zero ExitError error is returned.

import runcmd

mut okcmd := runcmd.new('sh', '-c', 'echo Hello, World!')
ok_out := okcmd.output()!
println(ok_out)
// Hello, World!

mut badcmd := runcmd.new('sh', '-c', 'echo -n Error! >&2; false')
bad_out := badcmd.output() or {
	if err is runcmd.ExitError {
		eprintln(err)
		exit(err.code())
	} else {
		// error starting process or handling I/O, see errno in err.code().
		panic(err)
	}
}
println(bad_out)
// &runcmd.ExitError{
//    state: exit status 1
//    stderr: 'Error!'
// }

fn (mut c Command) combined_output() !string

combined_output runs the command and returns its combined stdout and stderr. Unlike output(), this function does not return ExitError on command failure.

Note: The order of lines from stdout and stderr is not guaranteed, since reading from the corresponding file descriptors is done concurrently.

import runcmd
mut cmd := runcmd.new('sh', '-c', 'echo Hello, STDOUT!; echo Hello, STDERR! >&2')
output := cmd.combined_output()!
println(output)
// Hello, STDOUT!
// Hello, STDERR!

fn (mut c Command) start() !int

start starts a specified command and does not wait for it to complete. Call wait() after start() has successfully completed to wait for the command to complete and release associated resources.

Note: .state field is not set after start() call.

fn (mut c Command) wait() !

wait waits to previously started command is finished. After call see the .state field value to get finished process identifier, exit status and other attributes. wait() will return an error if the process has not been started or wait has already been called.

fn (mut c Command) release() !

release releases all resources assocuated with process.

fn (c Command) stdin() !WriteFd

stdin returns an open file descriptor associated with the standard input stream of the child process. This descriptor is write-only for the parent process.

fn (c Command) stdout() !ReadFd

stdout returns an open file descriptor associated with the standard output stream of the child process. This descriptor is read-only for the parent process.

fn (c Command) stderr() !ReadFd

stderr returns an open file descriptor associated with the standard error stream of the child process. This descriptor is read-only for the parent process.

struct ExitError {
pub:
	state  ProcessState
	stderr string
}

fn (e ExitError) code() int

code returns an exit status code of a failed process.

fn (e ExitError) msg() string

msg returns message about command failure.

struct Process {
pub:
	// Absolute path to the executable.
	path string

	// Arguments that will be passed to the executable.
	argv []string

	// Environment variables that will be applied to the child process.
	env map[string]string

	// Working directory for the child process.
	dir string

	// The pre_* and post_* fields store the functions that will be executed, respectively:
	// - before fork() call in the parent process;
	// - after fork() call in the parent process;
	// - after fork() call and before execve() call in the child process.
	pre_fork  []ProcessHookFn
	post_fork []ProcessHookFn
	pre_exec  []ProcessHookFn
mut:
	pid int = -1
}

fn (mut p Process) start() !int

start starts new child process by performing fork(3p) and execve(3p) calls. Return value is the child process identifier.

fn (p &Process) pid() int

pid returns the child process identifier. -1 is returned if process is not started.

fn (p &Process) wait() !ProcessState

wait waits for process to change state and returns the ProcessState.

fn (p &Process) signal(sig os.Signal) !

signal sends the sig signal to the child process.

fn (p &Process) kill() !

kill sends SIGKILL to the child process.