Spark HBase – possibly transient zookeeper

 

While trying to connect HBase with spark, we got an issue with zookeeper saying that possibly transient zookeeper.

To fix this issue copy hbase-site.xml file into spark conf directory across all NodeManager nodes.

ImportError: No module named ‘boto3’

 

I got this error while I am trying to use boto3 module in python.

The main reason for this error is, there is no boto3 module installed.

To install boto3 user following command:

sudo pip3 install boto3

One thing you need to observe here is to use pip3 to install boto3.

If pip3 not installed in your node user following command to install it:

sudo apt-get install python3-pip

Nginx – Host/serve an Android apk file

 

To host/serve Android apk with Nginx following changes are required:

  • Adding mime type in /etc/nginx/mime.types
  • Explicitly adding header *.apk in site conf

Adding mime type in /etc/nginx/mime.types:

Add following line in mime.types file

application/vnd.android.package-archive apk;

Explicitly adding header *.apk in site conf: 

Update following configuration in your site conf file

location ~* \.(apk)$ {
  ......
  add_header Content-Type application/vnd.android.package-archive;
  ......
}

Finally restart Nginx.

sudo service nginx restart

 

Lists/Remove all empty files in a directory

 

To list all empty files in a directory use follwoing command:

for file in `ls`; do if [[ ! -s $file ]]; then echo $file; fi; done

To remove all empty files in a directory use following commad:

for file in `ls`; do if [[ ! -s $file ]]; then echo $file; rm $file; fi; done

ftp – sync data from remote ftp server

 

To sync data from remote ftp server install ncftp with following command:

apt-get install ncftp

Now use following command to sync data from remote ftp server:

ncftpget -R -T -v -u userName -p 'password' ftp_hostname "directoryPathInFtpServer" "directoryPathInLocalMachine"

Gitlab backup

 

To backup Gitlab data use following command:

gitlab-rake gitlab:backup:create

This command will backup backup date in  path /var/opt/gitlab/backups/

To exclude any directories from backup use following command:

gitlab-rake gitlab:backup:create SKIP = db,uploads

Above command will exclude db & upload directories.

Get gitlab version number from Gitlab cli

 

To get Gitlab & other dependent modules version numbers use the following command:

gitlab-rake gitlab:env:info

The output will include all details like Gitlab version, git version, git shell version, OS version, Ruby version, Gem version, Bundler version, Rake version, Redis version, Sidekiq version & etc…

Gitlab with non-standerd ssh port

 

To user non standerd ssh port with gitlab use following procedure:

Open /etc/gitlab/gitlab.rb and add following line:

gitlab_rails['gitlab_shell_ssh_port'] = 4321

where 4321 is ssh port.

Make sure that you enabled this port in /etc/ssh/sshd_config.

Reconfigure & restart gitlab-ctl with following commands

gitlab-ctl restart

gitlab-ctl restart

Ubuntu 16.04 – Persisting IPTABLE Rules

 

To persist iptable rules installs iptables-persistent module.

sudo apt-get install iptables-persistent

Now you make changes to iptables & run following command to persist them:

sudo netfilter-persistent save
sudo netfilter-persistent reload

To view saved rules check in directory /etc/iptables/

Nginx – Deploy SSL certificate

 

These steps are used to to deploy SSL certificate issued by digicert.

Before starting make sure that you download ssl certificate issued by digicert.

Now create single file by concatenating  your_domain_com.crt and DigiCertCA.crt, name it as chained.crt

You can use following command to generate chained.crt

$ cat your_domain_com.crt DigiCertCA.crt > chained.crt

chained.crt file is used as ssl_certificate in nginx in this case.

Now configure nginx virtual hosts file for the website you wanted to secure as shown below:

server {

 listen 443 ssl default_server;
 ssl_certificate    /etc/nginx/ssl/chained.crt
 ssl_certificate_key    /etc/nginx/ssl/your_domain_name.key;

 server_name your.domain.com;

 access_log /var/log/nginx/access.log;
 error_log /var/log/nginx/error.log;

 location / {
  root   /home/www/public_html/your.domain.com/public/;
  index  index.html;
}

}

Finally restart nginx.

 

$ service nginx restart

After restarting your website can be accessed with https.