所用的软件版本：

spark2.3.0

hbase1.4.6

IDEA2019.1

输入数据源是多行的json格式的数据， 利用如下代码进行处理：

val session = SparkSession.builder().getOrCreate()
import session.implicits._
val DF = session.read.json("e:/test.json")
val res = DF.rdd.map(x => (x(11), x)).map(x => (x._2, x._1))
res.sortByKey(false)

DF是sql.DataFrame类型，DF.rdd直接转换成RDD[row]类型，然后直接利用val res = DF.rdd.map(x => (x(11), x)).map(x => (x._2, x._1))，转换成Tuple2(key, value)格式，利用sortByKey进行排序，但是却弹出报错"value sortByKey is not a member of org.apache.spark.rdd.RDD[Any, org.apache.spark.sql.Row]"，照着提示说明，key不能是Any类型，因此需要自定义一个key进行解决。

根据https://codeday.me/bug/20181209/445475.html和https://blog.csdn.net/hongxingabc/article/details/81638011文章的提示

// Generate pair RDD neccesary to call groupByKey and group it

    val key: RDD[((String, String, String), Iterable[Record])] = rs.keyBy(r => (r.day, r.city, r.kind)).groupByKey

// Once grouped you need to sort values of each Key

    val values: RDD[((String, String, String), List[Record])] = key.mapValues(iter => iter.toList.sortBy(_.prize))

首先要构造一个自定义class:

  case class result( cover:String, comment_count:BigInt, description:String, download_url:String, play_url:String, user_id:String, user_name:String, user_sex:String, video_create_time:Timestamp, video_id:String, like_count:BigInt, view_count:BigInt)

然后根据这个类，将输入的DF转换成result类型

val lines = DF.as[result].rdd

接着自定义key:

val key: RDD[(String, Iterable[result])] = lines.keyBy(x => x.video_id).groupByKey()
val values: RDD[(String, List[result])] = key.mapValues(iter => iter.toList.sortBy(_.view_count)(Ordering[BigInt].reverse).take(1))
val res = values.map(line => line._2).map(x => (x.head.view_count, x))
res.sortByKey(false).map(x => (x._1, x._2.head.play_url, x._2.head.video_create_time)).collect.take(10).foreach(println)

这样就不会报上述的错误！

做好记录，供以后参考！！

最后

以上就是孝顺鲜花为你收集整理的解决value sortByKey is not a member of org.apache.spark.rdd.RDD[Any, org.apache.spark.sql.Row]报错！！做好记录，供以后参考！！的全部内容，希望文章能够帮你解决解决value sortByKey is not a member of org.apache.spark.rdd.RDD[Any, org.apache.spark.sql.Row]报错！！做好记录，供以后参考！！所遇到的程序开发问题。

如果觉得靠谱客网站的内容还不错，欢迎将靠谱客网站推荐给程序员好友。