Initial Cleanup

Drop the tables from the Hive metadata with commands as shown here in Hue UI. Drop any other additional tables if present:

drop table customer;
drop table address;
drop table contacts;

Stop the dfs service (stop-dfs.sh) and clean up the Hadoop storage by formatting the Hadoop NameNode with the following command:

${HADOOP_HOME}/bin/hdfs namenode -format

Create new Hadoop directories with the following commands:

hdfs dfs -mkdir -p /datalake/raw/customer
hdfs dfs -mkdir -p /datalake/raw/address
hdfs dfs -mkdir -p /datalake/raw/contact

Remove the topics from Kafka servers to ensure we start clean, by using the following commands:

  1. Get a list of all the topics:
${KAFKA_HOME}/bin/kafka-topics.sh --list --zookeeper 0.0.0.0:2181
  1. For every topic, issue the following delete command (except for topic names prefixed with _):
${KAFKA_HOME}/bin/kafka-topics.sh --delete --topic <topic-name> --zookeeper 0.0.0.0:2181
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.145.202.27