溫馨提示×
您好，登錄后才能下訂單哦！
點擊 登錄注冊 即表示同意《億速云用戶服務條款》
您好，登錄后才能下訂單哦！
這篇文章主要介紹“hadoop常用操作命令有哪些”，在日常操作中，相信很多人在hadoop常用操作命令有哪些問題上存在疑惑，小編查閱了各式資料，整理出簡單好用的操作方法，希望對大家解答”hadoop常用操作命令有哪些”的疑惑有所幫助！接下來，請跟著小編一起來學習吧！
dfsadmin命令
report各幾點使用狀態
[hdfs@hd3 ~]$ hadoop dfsadmin -report
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
Configured Capacity: 768761769165 (715.97 GB)
Present Capacity: 739275145216 (688.50 GB)
DFS Remaining: 728906240000 (678.85 GB)
DFS Used: 10368905216 (9.66 GB)
DFS Used%: 1.40%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
-------------------------------------------------
Datanodes available: 4 (4 total, 0 dead)
Live datanodes:
Name: 10.202.11.20:50010 (hd1)
Hostname: hd1
Rack: /default
Decommission Status : Normal
Configured Capacity: 137276063744 (127.85 GB)
DFS Used: 3119067136 (2.90 GB)
Non DFS Used: 6016712704 (5.60 GB)
DFS Remaining: 128140283904 (119.34 GB)
DFS Used%: 2.27%
DFS Remaining%: 93.34%
Last contact: Fri Jul 25 11:20:55 CST 2014
Name: 10.202.11.21:50010 (hd2)
Hostname: hd2
Rack: /default
Decommission Status : Normal
Configured Capacity: 64706464973 (60.26 GB)
DFS Used: 2280718336 (2.12 GB)
Non DFS Used: 2592439501 (2.41 GB)
DFS Remaining: 59833307136 (55.72 GB)
DFS Used%: 3.52%
DFS Remaining%: 92.47%
Last contact: Fri Jul 25 11:20:55 CST 2014
Name: 10.202.11.22:50010 (hd3)
Hostname: hd3
Rack: /default
Decommission Status : Normal
Configured Capacity: 226409611264 (210.86 GB)
DFS Used: 2314543104 (2.16 GB)
Non DFS Used: 7515787264 (7.00 GB)
DFS Remaining: 216579280896 (201.71 GB)
DFS Used%: 1.02%
DFS Remaining%: 95.66%
Last contact: Fri Jul 25 11:20:55 CST 2014
Name: 10.202.11.23:50010 (hd4)
Hostname: hd4
Rack: /default
Decommission Status : Normal
Configured Capacity: 340369629184 (316.99 GB)
DFS Used: 2654576640 (2.47 GB)
Non DFS Used: 13361684480 (12.44 GB)
DFS Remaining: 324353368064 (302.08 GB)
DFS Used%: 0.78%
DFS Remaining%: 95.29%
Last contact: Fri Jul 25 11:20:55 CST 2014
safemode進入和離開安全模式
查看模式
[hdfs@hd2 ~]$ hadoop dfsadmin -safemode get
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
Safe mode is OFF
進入安全模式
[hdfs@hd2 ~]$ hadoop dfsadmin -safemode enter
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
Safe mode is ON
nodename在啟動時會處在安全模式，用這條命令可確保啟動完成后不離開安全模式。也可以指定dfs.sfaemode.threshold.pct設置>1
離開安全模式
[hdfs@hd2 ~]$ hadoop dfsadmin -safemode leave
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
Safe mode is OFF
在執行某條命令時臨時先退出安全模式，前面寫：
[hdfs@hd2 ~]$ hadoop dfsadmin -safemode wait
Fsck檢查HDFS文件監控狀況
檢查某個集群文件系統
[hdfs@hd1 ~]$ hadoop fsck /
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
Connecting to namenode via http://hd1:50070
FSCK started by hdfs (auth:SIMPLE) from /10.202.11.20 for path / at Tue Aug 19 14:35:53 CST 2014
....................................................................................................
....................................................................................................
....................................................................................................
.............................................Status: HEALTHY
Total size: 5120355488 B
Total dirs: 176
Total files: 345
Total blocks (validated): 364 (avg. block size 14066910 B)
Minimally replicated blocks: 364 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 2.6923077
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 4
Number of racks: 1
FSCK ended at Tue Aug 19 14:35:53 CST 2014 in 86 milliseconds
The filesystem under path '/' is HEALTHY
查詢特定文件的數據塊和分佈節點
[hdfs@hd2 ~]$ hadoop fsck /user/gc/hdfs_file -files -blocks -racks
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
Connecting to namenode via http://hd1:50070
FSCK started by hdfs (auth:SIMPLE) from /10.202.11.21 for path /user/gc/hdfs_file at Thu Sep 04 10:15:25 CST 2014
/user/gc/hdfs_file 19 bytes, 1 block(s): OK
0. BP-2070684995-10.202.11.20-1405329506510:blk_9027993399902156289_85985 len=19 repl=3 [/default/10.202.11.22:50010, /default/10.202.11.20:50010, /default/10.202.11.23:50010]
Status: HEALTHY
Total size: 19 B
Total dirs: 0
Total files: 1
Total blocks (validated): 1 (avg. block size 19 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 3.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 4
Number of racks: 1
FSCK ended at Thu Sep 04 10:15:25 CST 2014 in 1 milliseconds
刪除文件
hadoop fs –rm –f /user/hive/xx
刪除文件夾
hadoop fs –rmr –f /user/hive/xx
copyFromLocal 複製本地文件到HDFS
[hdfs@SCQHADoopN01 data]$ hadoop fs -copyFromLocal /data/copyToLocal_gc /user/hive/warehouse/test/part-m-00000-gc
[hdfs@SCQHADoopN01 data]$ hadoop fs -cat /user/hive/warehouse/test/part-m-00000-gc
1,abc
1,haha
copyToLocal 複製hdfs文件到本地
[hdfs@SCQHADoopN01 data]$ hadoop fs -copyToLocal hdfs://nameservice1/user/hive/warehouse/test/part-m-00000 /data/copyToLocal_gc
[hdfs@SCQHADoopN01 data]$ cd /data/
[hdfs@SCQHADoopN01 data]$ ll
total 16
-rw-r--r-- 1 hdfs hdfs 13 Nov 5 15:47 copyToLocal_gc
-rw-rw-r-- 1 hdfs hdfs 19 Aug 28 14:11 hdfs_file
drwxr-xr-x 3 root root 4096 Aug 28 08:52 mapred
drwx------ 3 hdfs hadoop 4096 Oct 10 14:01 nn
[hdfs@SCQHADoopN01 data]$ more copyToLocal_gc
1,abc
1,haha
到此，關于“hadoop常用操作命令有哪些”的學習就結束了，希望能夠解決大家的疑惑。理論與實踐的搭配能更好的幫助大家學習，快去試試吧！若想繼續學習更多相關知識，請繼續關注億速云網站，小編會繼續努力為大家帶來更多實用的文章！
免責聲明：本站發布的內容（圖片、視頻和文字）以原創、轉載和分享為主，文章觀點不代表本網站立場，如果涉及侵權請聯系站長郵箱：is@yisu.com進行舉報，并提供相關證據，一經查實，將立刻刪除涉嫌侵權內容。
