Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] SparkSql for Geomesa Hbase
Hello,

@Emilio, you were right again. Going back to Spark 2.2.0 solved the problem.
I read the upgrade guide for 2.1.0. Can I use it already or is it not stable right now?

I also saw that there exists a tutorial that suggests getting the 2.1.0-SNAPSHOT for GeoMesa HBase from here:
But as far as I can see, this file is not available there.

Thanks,

Martin


Am Mo., 15. Okt. 2018 um 17:11 Uhr schrieb Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx>:
Hello,

The issue comes from a spark version mismatch. In geomesa 2.0.2, I believe we only support spark 2.2. In current master (2.1.0-SNAPSHOT), we've updated to spark 2.3, while maintaining partial compatibility with 2.2 (we will offer full compatibility before release).

Generally we don't recommend mixing versions of geomesa in a single workflow, but you could try just using the current master hbase-spark-runtime and it might work. Longer term, you could upgrade all components to geomesa 2.1.0, or downgrade the spark version on your cluster.

Thanks,

Emilio

On 10/15/2018 10:56 AM, Martin Kellner wrote:
Hey, 

I got another question on Geomesa Hbase.
Ingesting data works perfectly now, thanks for your help so far.

Now I want to write some queries for Spark (in spark-shell).
Geomesa list lots of SparkSql functions:

I tried this query:
"select * from table where st_contains(st_makeBBOX(0.0, 0.0, 90.0, 90.0), geom)"
And I get this error:
java.lang.NoSuchMethodError: org.apache.spark.sql.execution.datasources.LogicalRelation.copy(Lorg/apache/spark/sql/sources/BaseRelation;Lscala/collection/Seq;Lscala/Option;)Lorg/apache/spark/sql/execution/datasources/LogicalRelation;

I am running spark-shell with these jars:

I also tried to add the jar for spark-sql like this:

Unfortunately that did not solve the problem. Do you have any ideas what could cause that error?

Thank you,
Martin





_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users

Back to the top