sbt安装与运行可.docxVIP

1.安装sbt

/download/

方法1：此方法速度极慢，需要耐心，经常断线

wget/sbt/sbt/releases/download/v1.9.9/sbt-1.9.9.tgz

解压到/usr/local/sbt

在sbt目录中建立文件sbtset.sh，内容如下

#!/bin/bash

SBT_OPTS=-Xms512M-Xmx1536M-Xss1M-XX:+CMSClassUnloadingEnabled-XX:MaxPermSize=256M

java$SBT_OPTS-jar`dirname$0`/sbt-launch.jar$@

chmod+xsbtset.sh

1.9.9版sbt会自动匹配scala的2.12.18版

sbtsbtVersion

方法2：此方法会更新软件包，不会断线

echodeb/scalasbt/debianallmain|sudotee/etc/apt/sources.list.d/sbt.list

echodeb/scalasbt/debian/|sudotee/etc/apt/sources.list.d/sbt_old.list

curl-sL/pks/lookup?op=getsearch=0x2EE0EA64E40A89B84B2DF73499E82A75642AC823|sudoapt-keyadd

sudoapt-getupdate

sudoapt-getinstallsbt

测试：

在/下建立一个实验目录，如下。

midir-p/example/sparkapp/src/main/scala

cd/example/sparkapp/src/main/scala

建立一个文件，SimpleApp.scala,内容如下：

测试：

/*SimpleApp.scala*/

importorg.apache.spark.SparkContext

importorg.apache.spark.SparkContext._

importorg.apache.spark.SparkConf

objectSimpleApp{

defmain(args:Array[String]){

vallogFile=file:///usr/local/spark/README.md//Shouldbesomefileonyoursystem

valconf=newSparkConf().setAppName(SimpleApplication)

valsc=newSparkContext(conf)

vallogData=sc.textFile(logFile,2).cache()

valnumAs=logData.filter(line=line.contains(a)).count()

valnumBs=logData.filter(line=line.contains(b)).count()

println(Lineswitha:%s,Lineswithb:%s.format(numAs,numBs))

}

}

在sparkapp目录下建立一个build.sbt文件，写入如下内容：

name:=SimpleProject

version:=1.0

scalaVersion:=2.13.13

libraryDependencies+=org.apache.spark%%spark-core%3.5.1

在spark中运行上面的程序：

spark-submit--classSimpleApp./target/scala-2.13/simple-project_2.13-1.0.jar21|grepLines