Spark HBase – possibly transient zookeeper

 

While trying to connect HBase with spark, we got an issue with zookeeper saying that possibly transient zookeeper.

To fix this issue copy hbase-site.xml file into spark conf directory across all NodeManager nodes.

Hadoop High Availability – FATAL ha.ZKFailoverController: Unable to start failover controller. Parent znode does not exist

 

Recently on working hadoop (version 2.5.1) cluster we got this issue while starting ZookeepreFailOverController(zkfc).

After debugging we found its due to missing/corrupted parent znode in zookeepre cluster.

This fix this we have used following command:

$ bin/hdfs zkfc -formatZK

 

We ran this command before starting hdfs.

After formating, zkfc started and everything started working smoothly.

HBase error there is a hole in the region chain

Recently while I am trying to restart HBase I got error message saying that “ERROR: There is a hole in the region chain between …… You need to create a new .regioninfo and region dir in hdfs to plug the hole”.

Following is the error message:

ERROR: There is a hole in the region chain between TestTable,2,1415170922328.3c1b2a210888171d142059912e2faba1. and TestTable,3,1415171044919.da852e5b0034a2ca83f6966280454b4a. You need to create a new .regioninfo and region dir in hdfs to plug the hole.
ERROR: Found inconsistency in table TestTable

To fix this issue you need to stop HBase and delete recover.edits folders for failing regions.

After that start HBase and run hbck which should succeed.