shell - Possible modification of nested for loop -
i newbie, , trying modify code below takes less time run. (right takes ages.) please or give suggestions, if possible. thank beforehand.
#!/bin/sh pheno in `cat /wrk/abc/composition/results/list.txt`; header=`head -1 /wrk/abc/composition/results/"$pheno"/meta_"$pheno".out` echo "pheno $header" > results.txt pheno in `cat /wrk/abc/composition/results/list.txt`; awk -v p="$pheno" \ 'nr == fnr{a[$1]; next}($3) in a{print p, $0}' \ list.txt \ /wrk/abc/composition/results/"$pheno"/meta_"$pheno".out \ >> results.txt done done
assuming list.txt line separated, here's same code simplified, no useless cat
s, (the for
loops swapped while read
s), , using cd
reduce unreadable long paths, followed notes. should little faster, , work same before, such was:
cd /wrk/abc/composition/results/ while read pheno ; { echo -n pheno; head -1 "$pheno"/meta_"$pheno".out ; } \ > results.txt while read pheno ; awk -v p="$pheno" \ 'nr == fnr{a[$1]; next}($3) in a{print p, $0}' \ list.txt \ "$pheno"/meta_"$pheno".out \ >> results.txt done < list.txt done < list.txt cd - mv /wrk/abc/composition/results/results.txt ./
the glaring error there 2 loops, 1 nested in other; both use same variable name ($pheno), both input same file (list.txt) -- surprisingly, sort of code may function correctly, despite being confusing. must cause slowdown, since inner loop runs awk
on same input file. if there 100 lines in list.txt, file might read 1,000,000 times.
then there's results.txt, inner loop appends data to, , outer loop overwrites every cycle. results.txt therefore winds being filled data last cycle.
Comments
Post a Comment