If you have a problem, if no one else can help... and if you can find them, maybe you can hire the bAsh-TEAM!

If you have a problem, if no one else can help... and if you can find them, maybe you can hire the bAsh-TEAMA different BASH shell scripting tutorial03.12.2009 @ Codebits 2009Thisdocumentisintellectualpropertyof PT Inovação, SA andshouldnotbeused without express written permission.

Schedule/AgendaA little about mePreliminary thoughtsPragmatic programming & Gluing stuffBASHSome characteristicsTricky snippets explainedIntroduction to “Hacking” web sites (tools, how-to)Some examples explainedExtra stuff (if we have time, I’ll guess we’ll have)References/Further reading03-12-20092

Warm-upRandom crap…03-12-20093

A little about meCo-founder of Radioactive Design portuguese demogroup in the 90’sExperience in “hacking” mostly in Linux, networking and protocols (messaging protocols@Layer 7, TCP@Layer 4)I love problems and to find their sources!Doing work for a “telco” R&D (PT Inovação), developing and managing projects and sometimes doing research LanguagesJava (mostly J2SE)PerlRuby / Ruby-on-RailsBASHSome C and if you dig a little you’ll find also Pascal and even Visual Basic 03-12-20094

Preliminary thoughtsPragmatic programmingDon’t try to reinvent the wheel, at least every week Be pragmatic:If you can reuse a solution, think about it!If you’re going to implement it, and if it really doesn’t matter what technology you should be using… then use the fastest/pragmatic approach; don’t implement it in your preferred language just because you like more that one03-12-20095

Gluing stuffMake it all work togetherSome call it gluing, other prefer “integration”… I just say make it all workIntegration can be made using well defined API’s or some “out-of-band glue”External Scripting analyzing logs, produced files, etc…Know your system!Be aware of the tools available in your OS, they will make your life a lot easier!03-12-20096

Typical Scripting Flow03-12-20097catfile.txt | egrep -v “[0-9]”| sort | uniq | xargstouch….

BASH: Bourne-Again ShellBASH it’s not just a simple shell, it’s not just a scripting language nor a simple applicationBASH is all about integrating software and controlling resources through scripting03-12-20099Once again, know your distribution!

Gluing with BASHPassing data/information all aroundData can be exchanged within hosts or between hosts, using sockets or any other messaging passing methodIn BASH programs communicate using:Standard streams (stdin, stdout, stderr)The programs exit/return codeSignalsAnd sometimes using environment variablesThe communication can be made by any kind of “out-of-band”protocol” (socket, some file, …)03-12-200910

Successvs FailureThereversedtruth: thebooleanupsidedownA successfulprogramterminationreturnstheexitcode 0; therefore “true” in BASH is 0Allabnormalprogramterminationreturns a valuediferentof 0 to the shell; therefore “false” in BASH iseverythingexcept 0Note thatinalmost 100% oflanguages, as inlogic, “false” isrepresentedby 0 and “true” byanyothervalue03-12-200911

(Some) BASH specialvariables$$Current PID of the running program$! PID of last child process started by the current shell$?Exit code of the last executed program!$ Arguments used in the last command$0 Name of current running script$1..$nFirst to n-th argument passed to the script03-12-200912

RedirectingStreamsStreamscanberedirectedYoucanmakestdoutpoint to some file descriptororyoucanmakestderrpoint to thesame FD usedbystdoutCautionwiththeorderofredirection: theredirectionismadenot as a wholebutfromleft to right> some_fileRedirectsstdoutto some_file2>&1Redirectsstderr to thesame FS usedby2>&-Redirectsstderr to null (/dev/null)> bla 2>&1isdifferentfrom2>&1 > bla03-12-200913

Handling signalsA BASH script, as any standard program, can catch and handle signals!You can do this to provide things such as configuration reloading or graceful exits03-12-200914$LOCK=/tmp/somefile.lock[ -f $LOCK ] && exit 0 trap "{ rm -f $LOCK ; exit 255; }" EXIT touch $LOCK || exit 255….

Paralell ExecutionMultiple programs can be started in the background, by appending “&” in the command line“wait” can be used to wait for child processes and therefore provide basic means of process synchronization; a PID argument can be given03-12-200915find /home/ -type d | do_some_weird_stuff&another_command&waitecho “yes, finally I can shutdown!! ” ; shutdown –hnow

SubshellsAnother shell can be instantiated within the current shell (the subshell has its own PID)A subshell is delimited by parenthesesYou can use a subshell to execute several programs/commands and “grab” the whole output (stdout/stderr) produced in the context of that subshell to further process it!03-12-200916(find /home/ -type d;ls /home/sergio) | grepgold

Tricky snippeTsGoing deep through and hacking with…03-12-200917

Introduction to “Hacking” Web sitesToolsA protocol analyzer is your friend (e.g. Wireshark / tshark) but it won’t deal with HTTPsYou can use Firefox plus some useful extensions no analyze HTTP(s) requests:Live HTTP headersTamperdataModify HeadersIn most Linux distributions you have HTTP clients that can be easily integrated with BASHGET/POSTwget (preferred)03-12-200918

Introduction to “Hacking” Web sitesHow-to (1/2)View the source of web pages, they’ll reveal much of the web site logic (and you’ll find many surprising things!)Sites often do validation in the client side using JavaScript => security problems,hidden or extended features03-12-200919

Introduction to “Hacking” Web sitesHow-to (2/2)Many parameters inside HTML are pure garbageSome sites demand HTTP requests to be made using POST while for other it’s indifferentSome sites do redirect on POST! So your client must either implement this behavior or else POST to the final URL03-12-200920

Fetchandsavemusicstreams03-12-200921pnumber[1]=2904; pname[1]="Antena3DanceClub”; pnumber[2]=1078; name[2]="AmbientaSons“; max=2; basedir="antena3“; totchilds=0; lastnchilds=-1for n in`seq 1 $max`; doecho "processing${pname[$n]}...”; mkdir -p "$basedir/${pname[$n]}“wmafiles=`GET -P "http://ww1.rtp.pt/multimedia/programa.php?hist=1&prog=${pnumber[$n]}" | egrep -o "mms://.*\.wma"|sort|uniq` for wma in $wmafiles; dofile=`basename "$wma"`filepath="$basedir/${pname[$n]}/$file“fprocs=`lsof$filepath2>- | wc -l 2>-`if [ "$fprocs" -gt 0 ]then echo "someone is accessing $file... bypassing.."elseecho "fetching$file..."nohupmimms -r "$wma" "$filepath" >- 2>&1 & ((totchilds++))fidonedoneecho "waiting for background processes to finish fetching all musics...“wait

Send SMS using a mobile operator’s site03-12-200922if [ $# -le 0 ]then echo "wrong syntax. use vod.sh dest message" exitfiuser=9100000; pass=“yourpassword”; dest=$1 ; msg=$2rm -f cookies.txtwget -O - --keep-session-cookies --save-cookies cookies.txt "http://www.vodafone.pt/main/myVodafone/" >-wget -O - --load-cookies cookies.txt --keep-session-cookies --save-cookies cookies.txt "--post-data=userid=$user&password=$pass&sru=https%3A%2F%2Fmy.vodafone.pt%2Fpm%2FSPMDispatcher.aspx%3FPMcmd%3D17%26userClass%3D10%26Guid%3D%7BB9DED956-B87F-4C24-852C-70A3BBBB0161%7D%26ReturnUrl%3Dhttps%253a%252f%252fmy.vodafone.pt%252fguest%252fhomepagePre.htm&fru=https%3A%2F%2Fmy.vodafone.pt%2Fguest%2FhomepagePre.htm&svc_id=myprodpub" “https://id.vodafone.pt/ucp//auth/login.asp?&ou=&crypt=0&prf=0&key=alias“ >-wget -O - --load-cookies cookies.txt --keep-session-cookies --save-cookies cookies.txt "http://mysms.vodafone.pt/rules/sms/sms_envio.asp" >-wget -O - --load-cookies cookies.txt --keep-session-cookies --save-cookies cookies.txt "--post-data=mydate3=&indicativo=91&telefone=&mensagem1=$msg&programado=nao&h2=1&phones=$dest%2F&prog=&dataf=12%2F29%2F2009+10%3A58%3A23+PM&agt=" \ "--referer=http://mysms.vodafone.pt/rules/sms/sms_envio.asp" \ “http://mysms.vodafone.pt/rules/sms/sms_envio.asp?submit=ok” >-

Accessingan online Bankandgetting data03-12-200923datainicial="2009-11-01"maxdias=$((365*5))hoje=`date +%Y-%m-%d`fundo="BPI Portugal"for i in`seq 0 $maxdias`; dodata=`date +%Y-%m-%d -d "$datainicial + $i day"`if [ "$data" == "$hoje" ]; thenbreakfireadvaldt< <(GET "http://www.bpiinvestimentos.pt/Fundos/QuadroCotacoesfundos.asp?opc=1&DataPesquisa=$data" | grep -A 2 "$fundo"| egrep -o "[0-9]{2}-[0-9]{2}-[0-9]{4}|[0-9]{1,2},[0-9]{1,}" |tr "\n" " "| awk '{print $2,$1}‘ )val=$(echo$val | tr "," ".")dt=$(echo$dt | tr "-" " "| awk '{print $3,$2,$1}' | tr " " "-")psql -q -h localhost -U postgres -c "deletefromcotacoeswherefundo='$fundo' anddata='$dt'" fundospsql -q -h localhost -U postgres -c "insertintocotacoes (fundo,data,valor) values ('$fundo','$dt',$val)" fundosdone

EXTRA STUFFIf we have time, we gonna take a bit of…03-12-200924

“here” / inlinedocumentsA heredocumentallowsthespecificationof a blockofthatcanbefed to a variable, to BASH functionsordirectly to the STDIN of a program03-12-200925multiline_variable=$(cat<<END_DELIMITERabEND_DELIMITER)echo “$multiline_variable”./some_app<<SEQUENCE_OF_APP_COMMANDSdo thisdo that…SEQUENCE_OF_COMMANDS

Processsubstitution“A kind of a reverse pipe”the output of a command (list) is redirected to a temporary file descriptor which can be used as the STDIN (directly or indirectly through an argument) of a program03-12-200926echo<( ls / ) /dev/fd/63cat<( ls / ) /home …# thefollowinglinewouldbe “hard” to do withpipes, unlessusing a subshelllike (ls /; ls /home) | grep “o”grep “o” <( ls / ) <( ls /home) /dev/fd/63:boot /dev/fd/63:cdrom /dev/fd/62:sergio /dev/fd/62:sergio …

References/Further readingAdvancedBash-ScriptingGuidehttp://tldp.org/LDP/abs/html/Linux Shell Scripting Tutorial - A Beginner's handbookhttp://bash.cyberciti.biz/guide/Main_PageSnipthttp://snipt.net/public/tag/bashBash by example, IBM developerWorkshttp://www.ibm.com/developerworks/library/l-bash.html03-12-200927

Thanks / ObrigadoBy:Sergio FreireMensagens e RedesConvergentes / Departamento de Redes e ProtocolosTel: +351 234 403609sergio-s-freire@ptinovacao.ptThisdocumentisintellectualpropertyof PT Inovação, SA andshouldnotbeused without express written permission.

If you have a problem, if no one else can help... and if you can find them, maybe you can hire the bAsh-TEAM!

More Related Content

Viewers also liked (19)

Similar to If you have a problem, if no one else can help... and if you can find them, maybe you can hire the bAsh-TEAM! (20)

Recently uploaded (20)

If you have a problem, if no one else can help... and if you can find them, maybe you can hire the bAsh-TEAM!