Username and password with wget/curl

 

To download a url with basic authentication we need to pass username & password.

To acheive this functionality with wget or curl observer following examples:

With curl:

$ curl -u 'username:password' 'http://www.example.com'

With wget:

$ wget --user='username' --password='password' 'http://www.example.com'

Wget with cookie:

Login to server & save cookie to a file:
$ wget --save-cookies cookiesFile.txt --post-data 'user=username&password=password' http://www.example.com/login.html

Download a page by using saved cookie file:
$ wget --load-cookies cookiesFile.txt -p http://www.example.com/somecontent.html

awk print Nth line after matching a pattern

 

By using awk we can print only Nth line after matching a pattern.

Lets say I have a file test.txt with following content and I want to extract 5 line after matching pattern.

$ cat test.txt

1
2
3
4
5
6
7
8
9

Use following awk code to extract Nth line:

Syntax:

$ awk 'c&&!--c;/pattern/{c=N}' file

where “pattern” is your input pattern and N is a line number to extract after matching pattern.

$ awk 'c&&!--c;/1/{c=5}' test.txt

Output:

6

In above example I am trying to extract 5th line after matching pattern, here my input pattern is 1 and 5th line after matching pattern is 6. So our output is 6.

bash – get previous command status

 

To get previous command status in bash use “$?” variable.

echo $? will return zero if previous command is success & if its non zero it indicates failure.

Example:

$ ls ; echo $?

Output of the above command is list of files, directories and next line it will return number.

We can use condition for check if previous command is success or failure:

any_command
if [ $? -eq 0 ] 
then
    echo SUCCESS
else
    echo FAIL
fi

awk – print last field

 

NF is built in variable in awk which stores Number of fields.

To get last field with awk use following example:

$ echo a b c | awk '{print $NF}'

Output:
c

By default awk will take any white space character(space, tab, …) to split.

To get last but one field with awk:

$ echo a b c | awk '{print $(NF-1)}'

Output:
b

To print only number of fields with awk:

$ echo a b c | awk '{print NF}'

Output:
3

Nmap – Scan for open ports

 

nmap is a network exploration tool and security/port scanner.

We can list all open ports using nmap for a given IP/Host.

Use following command to list all open ports:

$ sudo nmap -v -sS 192.168.1.1

Use following command to scan specific ports:

$ nmap -sS -O -p80,8080 192.168.1.1

To can specific range of ports(scan from port 80 to 100):

$ sudo nmap -sS -O -p80-100 192.168.1.1

mutt – send mails with attachments

 

By using mutt we can send mails with attachments from command line.

Install mutt:

$ apt-get install mutt

Send mail with attachment:

$ mutt -s "PFA" user@example.com -a attachment.txt < body.txt

or

$ echo "body text here" | mutt -s "PFA" user@example.com -a attachment.txt

Send mail without attachment:

$ mutt -s "PFA" user@example.com < body.txt

or

$ echo "body text here"  | mutt -s "PFA" user@example.com

awk/sed print from line number to end of file

I have file with 10 lines where content of first line in 1, second line 2, third line is 4 …… and tenth line is 10.

Here I will show some operations related to read read in lines based their line numbers.

Print from line number 5 to end of file:

$ awk 'NR>=5' inputFile
or
$ sed '1,4d' inputFile

Print from line number 4 to 9 or print lines between linen number 3 and 10:

$ awk 'NR>=4 && NR<=9'  inputFile
or
$ sed -n '4,9p' inputFile

Print all even lines numbers:

$ awk '{if(NR%2 == 0) {print $1}}' inputFile

Print all odd lines numbers:

$ awk '{if(NR%2 != 0) {print $1}}' inputFile

bash loop though multi line variable

 

I have a multi line variable in my bash script and wanted to iterate through each line of a variable.

To do this task I am using while loop.

Following is my variable:

var="a
b
c
d
e"

As I mentioned to loop through my multi line variable(var) I am using while as shown below:

while read -r line
do
  echo "line: $line"
done <<< "$var"

Just carefully observer last line in above code, here I am using three less than symbols(<<<) and variable in double quotes.

Output:
line: a
line: b
line: c
line: d
line: e

Above example only works with bash.

bash variable store and print multiple lines with new line

Bash supports string multi line output in a variable, also it allows to print multi line output with new line.

To store multi line output in a variable:

VAR=$(ls)

Above command will store ls outputs in VAR.

In general we will use echo to print content of variable. Here I will use use echo in two different forms and observe the output.

echo command without new line:

$ echo $VAR

Output:

t tmp tmp1 t.py

echo command with new line:

$ echo "$VAR"

t
tmp
tmp1
t.py

Just observe difference between above two commands, echo without double quotes given output without new lines and echo with double quotes given output with new line.  So whenever you store a multi line text in a variable use with double quotes.