Tag: iftop

Top helpful Linux commands for beginners

Top helpful Linux commands for beginners

Useful Command Line Keyboard Shortcuts

The following keyboard shortcuts are incredibly useful and will save you loads of time:

  • CTRL + U – Cuts text up until the cursor.
  • CTRL + K – Cuts text from the cursor until the end of the line
  • CTRL + Y – Pastes text
  • CTRL + E – Move cursor to end of line
  • CTRL + A – Move cursor to the beginning of the line
  • ALT + F – Jump forward to next space
  • ALT + B – Skip back to previous space
  • ALT + Backspace – Delete previous word
  • CTRL + W – Cut word behind cursor
  • CTRL + Insert – Copy selected text
  • Shift + Insert – Pastes text into terminal

sudo !!

If you don’t know it yet, sudo (super user do) is the prefix to run a command with an elevated privilege (like Run As Administrator in Windows).

sudo !! is a useful trick to save you from retyping the previous command that is “permission denied”.

For example, imagine you have entered the following command:

$ apt-get install ranger

The words “Permission denied” will appear unless you are logged in with elevated privileges.

Now, instead of typing this

$ sudo apt-get install ranger

Just type…

$ sudo !!

… and the previous denied command will be executed with sudo privileges.

Pausing Commands And Running Commands In The Background

I have already written a guide showing how to run terminal commands in the background.

So what is this tip about?

Imagine you have opened a file in nano as follows:

$ sudo nano abc.txt

Halfway through typing text into the file you realise that you quickly want to type another command into the terminal but you can’t because you opened nano in foreground mode.

You may think your only option is to save the file, exit nano, run the command and then re-open nano.

All you have to do is press CTRL + Z and the foreground application will pause and you will be returned to the command line. You can then run any command you like and when you have finished return to your previously paused session by entering “fg” into the terminal window and pressing return.

An interesting thing to try out is to open a file in nano, enter some text and pause the session. Now open another file in nano, enter some text and pause the session. If you now enter “fg” you return to the second file you opened in nano. If you exit nano and enter “fg” again you return to the first file you opened within nano.

Use nohup To Run Commands After You Log Out Of An SSH Session

The nohup command is really useful if you use the ssh command to log onto other machines.

So what does nohup do?

Imagine you are logged on to another computer remotely using ssh and you want to run a command that takes a long time and then exit the ssh session but leave the command running even though you are no longer connected then nohup lets you do just that.

For example, if I started downloading a large file on a remote host using ssh without using the nohup command then I would have to wait for the download to finish before logging off the ssh session and before shutting down the laptop.

To use nohup all I have to type is nohup followed by the command as follows:

$ nohup wget http://mirror.is.co.za/mirrors/linuxmint.com/iso//stable/17.1/linuxmint-17.1-cinnamon-64bit.iso &

Running A Linux Command ‘AT’ A Specific Time

The ‘nohup’ command is good if you are connected to an SSH server and you want the command to remain running after logging out of the SSH session.

Imagine you want to run that same command at a specific point in time.

The ‘at’ command allows you to do just that. ‘at’ can be used as follows.

$ at 10:38 PM Fri
at> cowsay 'hello'
at> CTRL + D

The above command will run the program cowsay at 10:38 PM on Friday evening.

The syntax is ‘at’ followed by the date and time to run.

When the at> prompt appears enter the command you want to run at the specified time.

The CTRL + D returns you to the cursor.

There are lots of different date and time formats and it is worth checking the man pages for more ways to use ‘at’.

Man Pages

Man pages give you an outline of what commands are supposed to do and the switches that can be used with them.

The man pages are kind of dull on their own. (I guess they weren’t designed to excite us).

You can however do things to make your usage of man more appealing.

$ export PAGER=most

You will need to install ‘most; for this to work but when you do it makes your man pages more colourful.

You can limit the width of the man page to a certain number of columns using the following command:

$ export MANWIDTH=80

Finally, if you have a browser available you can open any man page in the default browser by using the -H switch as follows:

$ man -H 

Note this only works if you have a default browser set up within the $BROWSER environment variable.

Use htop To View And Manage Processes

Which command do you currently use to find out which processes are running on your computer? My bet is that you are using ps and that you are using various switches to get the output you desire.

Install htop. It is definitely a tool you will wish that you installed earlier.

htop provides a list of all running processes in the terminal much like the file manager in Windows.

You can use a mixture of function keys to change the sort order and the columns that are displayed. You can also kill processes from within htop.

To run htop simply type the following into the terminal window:

$ htop

Navigate The File System Using ranger

If htop is immensely useful for controlling the processes running via the command line then ranger is immensely useful for navigating the file system using the command line.

You will probably need to install ranger to be able to use it but once installed you can run it simply by typing the following into the terminal:

$ ranger

The command line window will be much like any other file manager but it works left to right rather than top to bottom meaning that if you use the left arrow key you work your way up the folder structure and the right arrow key works down the folder structure.

It is worth reading the man pages before using ranger so that you can get used to all keyboard switches that are available.

Cancel A Shutdown

So you started the shutdown either via the command line or from the GUI and you realised that you really didn’t want to do that.

$ shutdown -c

Note that if the shutdown has already started then it may be too late to stop the shutdown.

Another command to try is as follows:

$ pkill shutdown

Killing Hung Processes The Easy Way

Imagine you are running an application and for whatever reason it hangs.

You could use ps -ef to find the process and then kill the process or you could use htop.

There is a quicker and easier command that you will love called xkill.

Simply type the following into a terminal and then click on the window of the application you want to kill.

$ xkill

What happens though if the whole system is hanging?

Hold down the alt and sysrq keys on your keyboard and whilst they are held down type the following slowly:

$ REISUB

This will restart your computer without having to hold in the power button.

Download Youtube Videos

Generally speaking most of us are quite happy for Youtube to host the videos and we watch them by streaming them through our chosen media player.

If you know you are going to be offline for a while (i.e. due to a plane journey or travelling between the south of Scotland and the north of England) then you may wish to download a few videos onto a pen drive and watch them at your leisure.

All you have to do is install youtube-dl from your package manager.

You can use youtube-dl as follows:

$ youtube-dl url-to-video

You can get the url to any video on Youtube by clicking the share link on the video’s page. Simply copy the link and paste it into the command line (using the shift + insert shortcut).

Download Files From The Web With wget

The wget command makes it possible for you to download files from the web using the terminal.

The syntax is as follows:

$ wget path/to/filename

For example:

$ wget http://sourceforge.net/projects/antix-linux/files/Final/MX-krete/antiX-15-V_386-full.iso/download

There are a large number of switches that can be used with wget such as -O which lets you output the filename to a new name.

In the example above I downloaded  AntiX Linux from Sourceforge. The filename antiX-15-V_386-full.iso is quite long. It would be nice to download it as just antix15.iso. To do this use the following command:

$ wget -O antix.iso http://sourceforge.net/projects/antix-linux/files/Final/MX-krete/antiX-15-V_386-full.iso/download

Downloading a single file doesn’t seem worth it, you could easily just navigate to the webpage using a browser and click the link.

If however you want to download a dozen files then being able to add the links to an import file and use wget to download the files from those links will be much quicker.

Simply use the the -i switch as follows:

$ wget -i /path/to/importfile

For more about wget visit http://www.tecmint.com/10-wget-command-examples-in-linux/.

Monitor your server performance with top command

When your machine is running slow and you want to find out what are the causes or what processes are eating a lot of resource, just type top in the command line:

$ top

This will bring up a real-time resource monitoring screen showing CPU usage, disk I/O bottlenecks, RAM usage of every running process and the whole system in general.

To show the full path of running processes, press “c”. To show only the processes’ names, press “c” again.

To show usage of each separate CPU core, press “1”. To show CPU usage as a total, press “1” again.

To kill a process, press “k”, type in the process’s id and press “ENTER”.

You can find more control keys by pressing “h” to bring up the help document. Go back to “top” screen by pressing “ESC”.

Monitor your disk free space with df command

To know how much free space are available on your disks, type

$ df -h

-h argument tells the df command to show the size in a human readable format (2.0GB instead of 2000000000).

Monitor your disk usage of each folder with du command

To know how much space is used up by each file/folder, type

$ du -h -c -d 1 ./*
  • -h tells the command to show the size in a human readable format  (2.0GB instead of 2000000000).
  • -c tells the command to show the total size of the folder, including subfolders and files (instead of showing that folder size only)
  • -d 1 tells the command to list the size of folders and subfolders up to child level 1. You can change it to 0 or 2 or whatever you are interested in.
  • ./* tells the command to do this to all the folders and files that are the children of current folder. You can change this param to any path you want, with or without wildcard (*).

Monitor your server traffic with iftop command

To monitor in real-time how much traffic is going in and out of your server, for each external IP, and each port, type the following command

$ iftop -n -B

If you get permission denied, try running it with sudo privilege.

  • -n tells the command not to try to translate the remote host to domain names but instead leave those as IPs.
  • -B tells the command to show traffic in bytes instead of bits.

If you don’t have iftop installed in your system, you can install it

For Ubuntu users

$ apt install iftop

For Centos/Fedora users

# Install epel-release if not yet available
$ yum install epel-release
# Install iftop
$ yum install iftop

Monitor log file in real-time with watch and tail command

If you have an application that logs to a file and you want to monitor that log in real-time, type the following command

$ watch tail /path/to/log/file.log

The watch command will watch for any update in the output of the tail command and keep printing the latest tail of that log file to the terminal console.

If you want to change the number of lines to print out, change the command to

$ watch tail -n 15 /path/to/log/file.log

Remember to replace 15 with the number of lines that you want.

List all running processes with ps command

To list all running process in your system, type the following command

$ ps aux

To filter the list with only the processes that you are interested in, add a grep command

$ ps aux | grep abc | grep xyz

To count how many processes are running, add a -c argument

$ ps aux | grep abc | grep xyz -c

List all connections with netstat command

To list all connections in your system, type the following command

$ netstat -naltp

To filter the result, add a grep command. For example, to filter all http connection (port 80) that is currently open (ESTABLISHED), type the following command

$ netstat -naltp | grep :80 | grep ESTABLISHED

To count how many results matches the filter, add -c argument

$ netstat -naltp | grep :80 | grep ESTABLISHED -c
5 tips to optimize traffic cost of your WordPress website

5 tips to optimize traffic cost of your WordPress website

Introduction

If you host your website on a cloud service, you may find out that your traffic costs even more than the hardware (CPU, RAM, HDD). If that’s the case, check the following tips on how to optimize your traffic cost.

Monitoring your traffic throughput

Before you optimize your traffic, you should have a monitoring tool for your network, so that you know how much traffic was optimized every time you do an experiment.

If you don’t know any traffic monitoring tool yet, try iftop. This tool can show you the real-time traffic throughput for every IP that is connected to your server. iftop is just like the top command in linux, but instead of monitoring the CPU or RAM like the top command, iftop monitors your network.

Install iftop

# fedora or centos
yum install iftop -y

# ubuntu or debian
$ sudo apt-get install iftop

Run iftop

$ sudo iftop -n -B

iftop console

After you run iftop, the iftop screen would appear like below

The first column is the IP of your current machine.

The second column are the IPs of the remote hosts that connect to your server. For each IP, the first row is the traffic that is sent to the remote host (notice the icon =>) and the second row is the traffic that is received from that remote host (hence the icon <=)

The third column is the real-time traffic amount that is sent or received, where the first sub-column is the average bandwidth per second in the last 2 seconds, the second sub-column is the average bandwidth per second in the last 10 seconds, and the third one is the average bandwidth per second in the last 40 seconds. Most of the time, I look at this third column.

Decide what to optimize

Deciding what cost to optimize is often depends on your hosting plan.

Some hosting plans do not charge for traffic. If that’s the case, the network optimizing is more about resource optimizing, where the lower the network traffic, the more requests your server can handle with the same CPU, RAM and network interface.

Some hosting plans do not charge for in going traffic and only charge for out going traffic. For example, Google Cloud only charge for out going traffic, and the rates differ by the destination zone. The cost for traffic between internal IP’s is a lot a lot cheaper than the traffic between external IP’s between zones, or regions, or continents.

By looking at the network traffic between your server and the remote IP’s and the traffic charging plan of your hosting service, you can decide which traffic to optimize first, and what can be left to be optimized later.

Tip #1. Changing to internal IP where possible

If your hosting plan rates are a lot expensive for external traffic than for internal traffic, try changing the external IP’s of your applications to internal IP’s can save you a lot of money.

For example, Google Cloud does not charge for traffic between internal IP’s within the same zone. Therefore, switching your servers to the same zone and configure them to talk to one another using the internal IP’s can save you a lot. Check your redis, memcached, kafka, rabbitmq, mysql or whatever services that can be run internally, make sure their configurations are optimized.

This action can reduce your traffic cost by 3-10 times.

Tip #2. Enable gzip

If you want to know more about gzip, check this post.

By enabling gzip, the traffic sent out will be compressed and therefore you can save a lot of network bandwidth.

This action can reduce your traffic cost by 3-5 times.

Tip #3. Using a free CDN service

If your website has a lot of images, try using a free CDN service to reduce the cost of serving images.

Believe it or not, the free CDN setup will take only 5 minutes and your traffic can be reduced by 5-10 times.

If you have a WordPress website, just install Jetpack plugin by WordPress.com and then turn on it’s Photon feature and your website is now powered with Jetpack’s free CDN service.

If you don’t want to use Jetpack Photon, you can always use CloudFlare or Incapsula CDN services, which are also free without any limitation on bandwidth or anything.

If your website has a lot of visitors in real-time, you can easily test the effects of the CDN by looking at the iftop console when Jetpack Photon is enabled and when it is disabled.

To read more on how to use a free CDN service on your website, click here.

Tip #4. Enable lazyload

When lazyload is enabled, the images on your website won’t be load until they are visible on the browser. Which means the images that stay at the end for your web pages won’t load at first. Then when the user scrolls the web page to where the images are located, the images will be loaded and shown for the user.

If you have a WordPress website, you can enable lazyload by installing the Lazy Load plugin.

Tip #5. Change your hosting service

I don’t know if this should be counted a tip. Anyways, if your current hosting service is charging too much for you traffic, consider changing to another hosting plan or service. Some hosting services do not charge for traffic such as BlueHost, GoDaddy and OVH.

However, even if you switch to a hosting service with free traffic, you can still consider applying the above tips as they can make your website perform better with less hardware resources.

 

How do these tips work for you? Let me know in the comment! 😀