|
Tom:
Thanks. This will save some time.
On 2/23/18 7:36 PM, Tom Kunicki wrote:
Hi, David.
I found this from ~1 year ago… If you're looking to
use Spark SQL or DataFrames this sample could be made more
concise.
Tom
From: Tom Kunicki <tkunicki@xxxxxxxx>
Subject: Re:
[geomesa-users] Geomesa Spark build :
ClassNotFoundException: org.apache.commons.io.Charsets
Date: March 6, 2017 at
10:28:51 AM EST
To: Geomesa User
discussions <geomesa-users@xxxxxxxxxxxxxxxx>
Hi, Jose.
There is an explicit Java API binding on the master branch
that will be released with GeoMesa 1.3.1. Here is a sample I
whipped up against my local test data set:
import org.apache.hadoop.conf.Configuration;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaSparkContext;
import org.geotools.data.Query;
import org.geotools.filter.text.ecql.ECQL;
import org.locationtech.geomesa.spark.api.java.*;
import java.util.HashMap;
import java.util.Map;
public class GeoMesaJavaSpark {
public static void main(String...
ags) throws Exception {
SparkConf conf = new SparkConf()
.setMaster("local[4]”)
.setAppName("Sample
Application");
conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer");
conf.set("spark.kryo.registrator", "org.locationtech.geomesa.spark.GeoMesaSparkKryoRegistrator");
JavaSparkContext jsc = new JavaSparkContext(conf);
Map<String, String> params
= new HashMap<>();
params.put("instanceId", "local");
params.put("zookeepers","localhost");
params.put("user", "root");
params.put("password", "secret");
params.put("tableName", "geomesa.gdelt");
JavaSpatialRDDProvider provider =
JavaGeoMesaSpark.apply((Map)params);
String filter = "BBOX(geom, -125, -24, -66,
50) AND dtg >= 2015-12-31T00:00:00Z";
Query query = new Query("gdelt", ECQL.toFilter(filter));
JavaSpatialRDD rdd =
provider.rdd(new Configuration(), jsc, params, query);
System.out.println(rdd.count());
System.out.println(rdd.first());
System.out.println(rdd.asGeoJSONString().first());
System.out.println(rdd.asKeyValueList().first());
System.out.println(rdd.asKeyValueMap().first());
}
}
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users
--
========= mailto:dboyd@xxxxxxxxxxxxxxxxx ============
David W. Boyd
VP, Data Solutions
10432 Balls Ford, Suite 240
Manassas, VA 20109
office: +1-703-552-2862
cell: +1-703-402-7908
============== http://www.incadencecorp.com/ ============
ISO/IEC JTC1 WG9, editor ISO/IEC 20547 Big Data Reference Architecture
Chair ANSI/INCITS TC Big Data
Co-chair NIST Big Data Public Working Group Reference Architecture
First Robotic Mentor - FRC, FTC - www.iliterobotics.org
Board Member- USSTEM Foundation - www.usstem.org
The information contained in this message may be privileged
and/or confidential and protected from disclosure.
If the reader of this message is not the intended recipient
or an employee or agent responsible for delivering this message
to the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication
is strictly prohibited. If you have received this communication
in error, please notify the sender immediately by replying to
this message and deleting the material from any computer.
|
