gnu bash script and general code optimisation tips

Over the years that I have been programming I had quite a few moments when I had to optimise code, so today I have decided to share how I do it, you might find this useful

# BASH script optimisation

Note: A lot of these points can be also applied to the next section

• Avoid forks and sub-shells, it might not look like much but it REALLY impacts your program's performance, like... By a lot, so avoid them
  • Prefer using built-in BASH commands (# Example 1) rather than calling external commands
    • Some builtins are faster than others (# Example 12)
  • Prefer using the -v syntax rather than using a sub-shell, capturing the output and saving it, by -v syntax I mean a command writing directly to the variable (# Example 2)
  • Prefer using native BASH rather than calling commands (# Example 3)
• Avoid looping, as in any interpreted programming language it's slow to loop in BASH
• Avoid complex commands (# Example 4)
  • Avoid complexity in general even if it sacrifices ease (# Example 5)
  • Be smart about the commands you call, call simpler ones (# Example 6)
• Less is more, if you're not using BASH features, why not stick to sh ? It's faster, or even use some other POSIX complient shell, for example DASH or KSH
• If your code is being sourced or in general, why not have a pre-processing or build step, for example let's say you have optional logging enabled by some environment variable, why not make that build-time, for example https://ari.lt/gh/baz does it, strip away comments and stuff
  • While you're at it, why not mangle names at build time to be shorter ? Shorter scripts from what I know run slightly faster as BASH has to read less and parse less
• Avoid disk I/O (# Example 7)
• Store data in variables rather than generating it over and over again for example BASH escapes $'\n', it gives a very slight performance boost (# Example 8)
• Prefer doing everything in one rather than one-by-one (# Example 9)

# General code optimisation

• Prefer compilation, transpilation or pre-evaluation over pure interpretation
  • Even if the transpilation is into bytecode, it doesn't matter, it'll still be faster than pure interpretation, for example python bytecode is faster than raw python
• Buffering is underrated, calling many syscalls is expensive, have a larger buffer instead ! (# Example 10)
• Prioritise simplicity over ease, abstractions often cause more complex code
• Use low level code, it's much faster than pure abstractions
  • Low level code gives you more control and is closer to hardware meaning is much faster than machine-generated assembly with preparation steps and things, you can do just what you want with low level code, although it's not easier, simple, but not easy
• Prefer smaller size, smaller assembly instructions and registers
• Find faster ways to do things, there always is at least one (https://stackoverflow.com/questions/1135679/does-using-xor-reg-reg-give-advantage-over-mov-reg-0)
• Prefer doing less for a similar result (# Example 11)

# Examples

# Example 1

1

x="$(cat -- /etc/passwd)"

Faster:

1

x="$(</etc/passwd)"

# Example 2

1
2
3
4

greet() { echo "Hello, $1"; }

x="$(greet 'ari')"
echo "$x"

Faster:

1
2
3
4
5
6
7
8
9

greet() {
    local -n _r="$1"
    shift 1

    printf -v _r "Hello, %s" "$1"
}

greet x 'ari'
echo "$x"

# Example 3

1
2

x="Hel o"
echo "$x" | sed 's/ /l/'

Faster:

1
2

x="Hel o"
echo "${x/ /l}"

# Example 4

1

printf '%s\n' 'hey'

Faster:

1

echo 'hey'

# Example 5

1
2
3
4
5

x=()

while read -r line; do
    x+=("$line")
done <file

Faster:

1

mapfile -t x <file

# Example 6

1

sed '1!d' file

Faster:

1

head -n1 file

# Example 7

1
2
3

id >/tmp/x
echo "Info: $(</tmp/x)"
rm -f /tmp/x

Faster:

1

echo "Info: $(id)"

# Example 8

1
2
3

for _ in $(seq 10000); do
    echo "Hello"$'\n'"world"
done

Faster:

1
2
3
4
5

nl=$'\n'

for _ in $(seq 10000); do
    echo "Hello${nl}world"
done

# Example 9

1
2
3

while read -r line; do
    echo "$line"
done <file

Faster:

1

echo "$(<file)"

# Example 10

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

format ELF64 executable 3
segment readable executable

_start:
    ;; 2 syscalls per char

    mov eax, 0
    mov edi, 0
    mov esi, buf
    mov edx, 1
    syscall

    test eax, eax
    jz .exit

    mov eax, 1
    mov edi, 1
    mov esi, buf
    mov edx, 1
    syscall

    jmp _start

.exit:
    mov rax, 60
    mov rdi, 0
    syscall

segment readable writable
    buf: rb 1

Faster:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

format ELF64 executable 3
segment readable executable

_start:
    ;; 2 syscalls per 1024 chars

    mov eax, 0
    mov edi, 0
    mov esi, buf
    mov edx, 1024
    syscall

    test eax, eax
    jz .exit

    mov edx, eax

    mov eax, 1
    mov edi, 1
    mov esi, buf
    syscall

    jmp _start

.exit:
    mov rax, 60
    mov rdi, 0
    syscall

segment readable writable
    buf: rb 1024

# Example 11

1
2
3
4
5

int x = 0;
x = 0;
x = 1;
x--;
x++;

Faster:

1

int x = 1;

# Example 12

1

content="$(cat /etc/passwd)"

Faster:

1

content="$(</etc/passwd)"

Faster:

1
2

mapfile -d '' content </etc/passwd
content="${content[*]%$'\n'}"

Faster:

1

read -rd '' content </etc/passwd

^ This exists with code 1, so just add a || : at the end if that's unwanted behaviour