Hadoop High Availability – FATAL ha.ZKFailoverController: Unable to start failover controller. Parent znode does not exist

 

Recently on working hadoop (version 2.5.1) cluster we got this issue while starting ZookeepreFailOverController(zkfc).

After debugging we found its due to missing/corrupted parent znode in zookeepre cluster.

This fix this we have used following command:

$ bin/hdfs zkfc -formatZK

 

We ran this command before starting hdfs.

After formating, zkfc started and everything started working smoothly.

Advertisements

Hadoop – Decommission datanode

 

Decommissioning is process to remove one or multiple datanodes from Hadoop.

To decommission a datanode you need to use following process:

Login to namenode host.

Add follwoing configuration in/home/guest/hadoop-2.5.1/etc/hadoop/hdfs-site.xml

<property>
 <name>dfs.hosts.exclude</name>
 <value>/home/guest/hadoop-2.5.1/etc/hadoop/decommission-nodes</value>
</property>

After adding dfs.hosts.exclude property you need to restart HDFS.

Then add datanode hostname that you want to remove inĀ /home/guest/hadoop-2.5.1/etc/hadoop/decommission-nodes

Now run follwoing command to start decommissioning datanode

$ hadoop-2.5.1/bin/hdfs dfsadmin -refreshNodes

This process will run for few minutes depending on data size in datanode, keep monitor decommissioning status in on http://namenode:50070

After finishing decommissioning you can remove that datanode.