Thursday, October 30, 2014

Loop on tail -f for dynamic processing

Lets say you get some interesting files:

for a in {000001..249510}; do wget --random-wait -U "Mozilla/5.0 (Windows NT 5.1; rv:10.0.1)" "http://oeis.org/A$a"; done

then you suspect you missed a few... 

You can open a terminal window (or background) this: 

for a in {000001..249510}; do ((! $(grep -c A$a files.txt))) && (echo $a >> missing_files.txt); done

... and since it will take a while in a new terminal you can "Loop on tail -f for dynamic processing"

tail -f missing_files.txt | while read a; do wget --random-wait -U "Mozilla/5.1 (Windows NT 5.0; rv:10.0.2)" "http://oeis.org/A$a"; done

No comments:

Post a Comment