Re: [geomesa-users] Tools/Utilities to analyze Geomesa query stats

[David Boyd <dboyd@xxxxxxxxxxxxxxxxx>]

Emilio:

   Thanks man.   This is cool stuff.
I got a good way through the process.  Now I am currently running 1.3.1 
if it makes a difference.

The analytics endpoint come up and I get the expected list of options.
I am able to set and retrieve the spark config map.

When I try to register my data store using:

> curl -d 'instanceId=rdk' -d 'zookeepers=bdpnode4:bdpnode5:bdpnode6' -d 
> 'tableName=CoalesceSearch' -d 'user=<myuser>' -d 'password=<mypass>' 
> -v http://bdpgeoserver:8181/geoserver/geomesa/analytics/ds/CoalesceSearch
I get back a 400 error - Bad request

However, in looking at my tomcat logs the spark library failed to load 
with this error:

> Nov 28, 2017 11:42:26 PM 
> org.apache.catalina.loader.WebappClassLoaderBase validateJarFile
> INFO: 
> validateJarFile(/opt/tomcat7/webapps/geoserver/WEB-INF/lib/spark_2.11-1.3.1-geomesa-assembly.jar) 
> - jar not loaded. See Servlet Spec 3.0, section 10.7.2. Offending 
> class: javax/servlet/Servlet.class
So I tried a simple query:

> Davids-MacBook-Pro:Documents BigDataDaddy$ curl --header 'Accept: 
> text/plain' --get --data-urlencode 'q=select 
> count(EventRecordset.GlobalEventId) from EventRecordset' 
> http://bdpgeoserver:8181/geoserver/geomesa/analytics/sql
> java.lang.NoClassDefFoundError: org/apache/spark/sql/types/DataType
>     at 
> org.locationtech.geomesa.web.analytics.AnalyticEndpoint$$anonfun$12.apply(AnalyticEndpoint.scala:107)
>     at 
> org.scalatra.ScalatraBase$class.org$scalatra$ScalatraBase$$liftAction(ScalatraBase.scala:270)
>     at 
> org.scalatra.ScalatraBase$$anonfun$invoke$1.apply(ScalatraBase.scala:265)
>     at 
> org.scalatra.ScalatraBase$$anonfun$invoke$1.apply(ScalatraBase.scala:265)
Which I assume is due to the jar file not loading.

So I tried the end point to access the Queries table. But since the 
datastore did not register I am assuming it will not work.

Any thoughts?

I assume where :alias shows up in the list of endpoints one should 
substitute the name of the datastore registered?



On 11/28/17 8:47 AM, Emilio Lahr-Vivaz wrote:
> Although it's not documented well (at all?), GeoMesa has a rest API 
> for reading the queries table:
>
> https://github.com/locationtech/geomesa/tree/master/geomesa-web/geomesa-web-data 
>
> http://www.geomesa.org/documentation/user/web_data.html
>
> You shouldn't need the spark jars from the documentation if you are 
> just accessing the query table. The documentation focuses on the 
> analytic endpoint, but there is another endpoint under 'stats' that 
> will let you query the queries table:
>
> https://github.com/locationtech/geomesa/blob/master/geomesa-web/geomesa-web-data/src/main/scala/org/locationtech/geomesa/web/analytics/StatsEndpoint.scala#L51 
>
>
> If you can't muddle through it, write back and I can provide better 
> instructions.
>
> Thanks,
>
> Emilio
>
> On 11/27/2017 08:04 PM, David Boyd wrote:
> > All:
> >
> >    Was browsing the queries table in my instance looking at the 
> > times.   I was curious if anyone had
> > any short utils to browse or download the queries table for my 
> > catalog.   I see the new query-rasterstats in
> > 1.3.4 but nothing to pull from that table.    I know I can do an scan 
> > from the accumulo shell and download the
> > results to post process or write a small accumulo query application, 
> > but I was wondering if someone had thrown
> > something together already.
>
> _______________________________________________
> geomesa-users mailing list
> geomesa-users@xxxxxxxxxxxxxxxx
> To change your delivery options, retrieve your password, or 
> unsubscribe from this list, visit
> https://dev.locationtech.org/mailman/listinfo/geomesa-users

--
========= mailto:dboyd@xxxxxxxxxxxxxxxxx ============
David W. Boyd
VP,  Data Solutions
10432 Balls Ford, Suite 240
Manassas, VA 20109
office:   +1-703-552-2862
cell:     +1-703-402-7908
============== http://www.incadencecorp.com/ ============
ISO/IEC JTC1 WG9, editor ISO/IEC 20547 Big Data Reference Architecture
Chair ANSI/INCITS TC Big Data
Co-chair NIST Big Data Public Working Group Reference Architecture
First Robotic Mentor - FRC, FTC - www.iliterobotics.org
Board Member- USSTEM Foundation - www.usstem.org

The information contained in this message may be privileged
and/or confidential and protected from disclosure.
If the reader of this message is not the intended recipient
or an employee or agent responsible for delivering this message
to the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication
is strictly prohibited.  If you have received this communication
in error, please notify the sender immediately by replying to
this message and deleting the material from any computer.