Inhaltsverzeichnis

Sammlung nützlicher Befehle
- General Tool Commands

Sammlung nützlicher Befehle

General Tool Commands

Development

fetch all git branches and prune remotely deleted branches

git remote update origin --prune

man git-remote

delete all files and directories in a Git work tree which are under gitignore (e.g. for removing builds)

git clean -Xdf

man git-clean man gitignore

get number of commits in a Git repository

## total number (in all branches)
git rev-list --all | wc -l
## number per committer
git shortlog -s -n --all

man git-rev-parse man git-shortlog

show predefined macros of a compiler

gcc -dM -E -x c /dev/null
g++ -dM -E -x c /dev/null
clang -dM -E -x c /dev/null
clang++ -dM -E -x c /dev/null

OpenSSL

show details for a certificate file

# certificate has PEM format (Pretty Enhanced Mail)
openssl x509 -in cert-file.pem -text -noout -fingerprint
# certificate has DER format (Distinguished Encoding Rules)
openssl x509 -inform der -in cert-file.cer -text -noout -fingerprint

man x509

show details for a private key file

openssl rsa -in privkey.pem -text -noout

man rsa

show details of a certificate signing request

openssl req -in csr.pem -noout -text

man req

extract certificate and private key from PKCS#12 file

openssl pkcs12 -in yourP12File.pfx -clcerts -nokeys -out publicCert.pem
openssl pkcs12 -in yourP12File.pfx -nocerts -out privateKey.pem

man pkcs12

create a self-signed certificate

openssl genrsa -out privKey.pem
openssl req -new -x509 -key privKey.pem -out cert.pem

man genrsa man req

AES-256-CBC benchmark

## with AES-NI (if available)
openssl speed -evp aes-256-cbc
## without AES-NI
OPENSSL_ia32cap="~0x200000200000000" openssl speed -evp aes-256-cbc
## multithread
openssl speed -multi 2 -evp aes-256-cbc

man speed

parse a file's ASN.1 content

openssl asn1parse -in someCertOrKeyOrOther.pem

man asn1parse

Network Stuff

test a proxy server

curl test.schrimpe.de -x <[protocol://][user:password@]proxyhost[:port]> | w3m -dump -T text/html
# for Squid running on localhost that's:
curl test.schrimpe.de -x 127.0.0.1:3128 | w3m -dump -T text/html

curl(1)

measure duration of HTTP request

curl https://test.schrimpe.de/ -w %{time_connect}:%{time_starttransfer}:%{time_total}

curl(1)

check whether a TCP connection can be established with a three second timeout

nc -w3 -z 192.168.0.1 80 ## return code zero on success

nc(1)

mDNS query

dig somehostname.local @224.0.0.251 -p 5353

dig(1)

verify DNSSEC chain of a domain

## if you don't have the root keys yet, obtain them first
unbound-anchor -a /usr/local/etc/unbound/root.key
## if the root key file is at the standard path of your unbound/ldns installation, the "-k" option can be omitted
drill -S -k /usr/local/etc/unbound/root.key kuehrmann.de
## if your resolver does not support DNSSEC, use a public one, e.g. Hurricane Electric
drill -S -k /usr/local/etc/unbound/root.key kuehrmann.de @74.82.42.42

drill(1) unbound-anchor(8)

check whether your DNS resolver discards invalid DNSSEC domains

dig www.dnssec-failed.org

dig(1)

check whether your DNS resolver uses QNAME minimisation

dig txt qnamemintest.internet.nl +short

dig(1)

scan all TCP ports of a host

nmap -p- somehost.example.com

nmap(1)

reverse DNS lookup of an IP subnet

nmap -R -sL 198.51.100.96/27
nmap -R -sL -6 2001:db8::43a1:89d0/124

nmap(1)

scan for responding hosts in an IP subnet

## probe TCP connect on Port 80 with five attempts for each host
nping 198.51.100.96/27
## probe ICMP with five attempts for each host
nping --icmp 198.51.100.96/27
## probe ICMP with one attempt for each host and 200ms between requests (and decrease verbosity)
nping -q --count 1 --delay 200ms --icmp 198.51.100.96/27

nping(1)

File Stuff

count lines of a certain file type recursively (e.g. C++ source files)

find /your/dir -name "*.cpp" -print0 | xargs -0 wc -l

find(1) xargs(1)

check recursively for broken symlinks

find /your/dir -type l ! -exec test -e {} \; -print

find(1)

delete recursively all files with a certain name (e.g. “.DS_Store”)

find /your/dir -type f -name .DS_Store -print -delete

find(1)

create checksums of directory tree and verify them

# create checksums of directory tree
rhash -r --md5 /backups/ > /tmp/backups.md5
rhash -r --crc32 /backups/ > /tmp/backups.sfv
# check one file containing MD5 hashes
rhash -c /tmp/backups.md5
# check all SFV, MD5 and SHA-256 files in directory tree
rhash -cr --crc-accept=sfv,md5,sha256 /tmp/directory/

rhash(1)

create a file with specific timestamps

touch -a -m -t 202612180130.09 fileName.ext
## -a = accessed
## -m = modified
## -t = timestamp - use [[CC]YY]MMDDhhmm[.ss] time format

touch(1)

handle sparse files

## create a 100 GiB sparse file
truncate -s 100G fileName.ext
## copy sparse files (works also from/to block devices; 4k should match the block size used within you sparse file)
dd if=inputSparse.ext of=outputSparse.ext bs=4096 conv=sparse

truncate(1) dd(1)

create a 7-ZIP archive encrypted with a password

7z a -p -mhe=on the_new_archive.7z -r dir_containing_stuff another_file.bin

7z(1)

view contents of ZIP or TAR files without complete extracting

### example of "Android Package Kit" which is actually a ZIP file
unzip -l SomeAndroidApp.apk
unzip -p SomeAndroidApp.apk META-INF/MANIFEST.MF
### example of an "Open Virtual Appliance" which is actually a TAR file
tar -tf SomeAppliance.ova
tar -xfO SomeAppliance.ova SomeAppliance.ovf

unzip(1) tar(1)

Disk and Partition Stuff

show information about partitions of disks (on Linux)

file -s /dev/sd*

disk images over SSH

## from a local disk send compressed image to remote host
dd if=/dev/sda bs=1048576 | pv | lz4 | ssh user@dest.example.com "dd of=/backup/basil-sda.img.lz4"
## to a local disk receive compressed image from remote host
ssh user@src.example.com "cat /backup/basil-sda.img.lz4" | lz4cat | pv | dd of=/dev/sda bs=1048576

Conversion Stuff

convert image file using ImageMagick

## make a big image small
convert big_input.tif -resize 256x256 small_output.png
## make an image grayscale and remove any transparency
convert input.png -alpha deactivate -colorspace Gray output.png
## create PNG with transparency from SVG, determine it's pixel dimensions from a DPI parameter
convert -background none -density 1200 input.svg output.png
## create PNG with transparency from SVG, specify it's pixel dimensions directly
convert -background none -size 512x512 input.svg output.png

convert(1) ImageMagick(1)

batch convert images in a directory: rename by creation date and resize to width

### for JPGs
exiftool -ext jpg '-FileName<CreateDate' -d %Y%m%d_%H%M%S%%-c.%%e .
mogrify -resize 375 -define preserve-timestamp=True *.JPG
### for PNGs
exiftool -ext png '-FileName<DateCreated' -d %Y%m%d_%H%M%S%%-c.%%e .
mogrify -resize 375 -define preserve-timestamp=True *.PNG
### Comments:
## the "%%-c" in the filename template is for a numerical suffix, in case of multiple images with the same date
## the "-define preserve-timestamp=True" is for keeping file attributes for timestamps
## for case-sensitive filesystems in the "mogrify" command the glob pattern must be adjusted

exiftool(1) mogrify(1) ImageMagick(1)

add rounded corners to an image (for app icon with rounded corners in the style of iPhone‘s SpringBoard)

### Adjust the corner radius (in pixel) to your needs.
### For a 1024x1024 image a value of 128 looks good.
RADIUS=128
 
### The whole ImageMagick command is quite complex. The resulting rounded image is written to a new file. 
### The cascade of parenthesized options defines the operations to cut out the transparent arcs in the corners.
convert AppIconRaw.png \( +clone -alpha extract \
  \( -size ${RADIUS}x${RADIUS} xc:black -draw "fill white circle ${RADIUS},${RADIUS} ${RADIUS},0" -write mpr:arc +delete \) \
  \( mpr:arc \) -gravity northwest -composite \
  \( mpr:arc -flip \) -gravity southwest -composite \
  \( mpr:arc -flop \) -gravity northeast -composite \
  \( mpr:arc -rotate 180 \) -gravity southeast -composite \) \
  -alpha off -compose CopyOpacity -composite AppIconRounded.png

convert(1) ImageMagick(1) ImageMagick - Command-line Processing ImageMagick - Command-line Options

Unix timestamp to human readable time

## with BSD date
date -r 1523210382
## with GNU date
date -d @1523210382

date(1) from BSD — date(1) from GNU

format XML

## prettify XML (proper indentation)
xmllint --format input_ugly.xml > output_pretty.xml
## uglify XML (remove unnecessary whitespace)
xmllint --noblanks input_pretty.xml > output_ugly.xml

xmllint(1)

C array of the content of some file

xxd -i someFile.bin

xxd(1)

change the character encoding of a file‘s content

## convert from "UTF-8" to "ISO-8859-1"
recode utf8...latin1 test.dat 
## autodetect and convert to "ISO-8859-1"
recode ..latin1 test.dat
## list available encodings available in your "recode" build
recode --list

recode(1) (more lightweight and probably already installed alternative: iconv(1))

occurrences of non-ASCII characters in a file‘s content (with GNU grep)

ggrep --color='auto' -P -n "[\x80-\xFF]" file.txt

occurrences of invalid UTF-8 characters in a file‘s content (this assumes you have an UTF-8 locale set)

grep -axv '.*' file.txt

some StackOverflow discussion

count how many time each byte value occurs in a file‘s content

recode latin1/..count-characters < input.dat

recode(1)

Process Stuff

execute a program in a background terminal

## create named session using the terminal emulator "screen"
screen -dMS some_session_name /usr/bin/command command_args
## show PID of the "screen" session
screen -lsS some_session_name | awk '/\.some_session_name\t/ {print $1}' | cut -d. -f1
## attach to the session
screen -rS some_session_name

show child PIDs of a PID

pgrep -P 56789

Bourne Again Shell

delete entries from the command history (e.g. to remove passwords)

## show the current history
history
## determine the entry number, in this example the desired entry was #501
history -d 501

avoid history recording for one interactive shell completely

unset HISTFILE

kuehrmann.de

Benutzer-Werkzeuge

Webseiten-Werkzeuge

Inhaltsverzeichnis

Sammlung nützlicher Befehle

General Tool Commands

Development

fetch all git branches and prune remotely deleted branches

delete all files and directories in a Git work tree which are under gitignore (e.g. for removing builds)

get number of commits in a Git repository

show predefined macros of a compiler

OpenSSL

show details for a certificate file

show details for a private key file

show details of a certificate signing request

extract certificate and private key from PKCS#12 file

create a self-signed certificate

AES-256-CBC benchmark

parse a file's ASN.1 content

Network Stuff

test a proxy server

measure duration of HTTP request

check whether a TCP connection can be established with a three second timeout

mDNS query

verify DNSSEC chain of a domain

check whether your DNS resolver discards invalid DNSSEC domains

check whether your DNS resolver uses QNAME minimisation

scan all TCP ports of a host

reverse DNS lookup of an IP subnet

scan for responding hosts in an IP subnet

File Stuff

count lines of a certain file type recursively (e.g. C++ source files)

check recursively for broken symlinks

delete recursively all files with a certain name (e.g. “.DS_Store”)

create checksums of directory tree and verify them

create a file with specific timestamps

handle sparse files

create a 7-ZIP archive encrypted with a password

view contents of ZIP or TAR files without complete extracting

Disk and Partition Stuff

show information about partitions of disks (on Linux)

disk images over SSH

Conversion Stuff

convert image file using ImageMagick

batch convert images in a directory: rename by creation date and resize to width

add rounded corners to an image (for app icon with rounded corners in the style of iPhone‘s SpringBoard)

Unix timestamp to human readable time

format XML

C array of the content of some file

change the character encoding of a file‘s content

occurrences of non-ASCII characters in a file‘s content (with GNU grep)

occurrences of invalid UTF-8 characters in a file‘s content (this assumes you have an UTF-8 locale set)

count how many time each byte value occurs in a file‘s content

Process Stuff

execute a program in a background terminal

show child PIDs of a PID

Bourne Again Shell

delete entries from the command history (e.g. to remove passwords)

avoid history recording for one interactive shell completely

Seiten-Werkzeuge