spark-shell でテキスト読み込み

読み込み用のテキストを用意

directories.txt

Documents
metastore_db
derby.log
Downloads
dev
anaconda3
Music
Pictures
Public
Templates
Videos
Desktop

読み込み用のRDD生成

spark-shell

scala> val dirRDD = sc.textFile("directories.txt")
dirRDD: org.apache.spark.rdd.RDD[String] = directories.txt MapPartitionsRDD[7] at textFile at <console>:24

空でないか確認

spark-shell

scala> dirRDD.isEmpty
res6: Boolean = false

リストのサイズをカウント

spark-shell

scala> dirRDD.count
res7: Long = 12

リストの中身を表示

spark-shell

scala> dirRDD.collect
res8: Array[String] = Array(Documents, metastore_db, derby.log, Downloads, dev, anaconda3, Music, Pictures, Public, Templates, Videos, Desktop)