Michael:
So you don't say what you are using for
visualization/display (thick client or web application) but the
core
issue is the same. The approach you describe is one good way of
approaching the problem. You may also
want to look at using Web Processing Service (WPS). GeoMesa has
support for that and will do the work
distributed across the cluster. I have had some success with the
Geomesa heat map, but have not tried
any other WPS functions.
The end problem for the visualization is two fold:
1. Getting the data from the database to client
2. Rendering the data so that it is something other than a blob.
Typically, even if you can accomplish #1 quickly, most front end
UI tools tend
to get overwhelmed with all the data. Hence server side
rendering. The harder
part is to render something other than a blob.
The root of the question is what task is the analyst trying to
solve with the data?
And what in that task requires all that data be visualized?
Typically, for other Big Data visualizations, you would run some
sort of analytic
server side that would reduce the data volume by limiting the
display to what is of
interest to the analyst. For example, on one project with AIS
(ship location data) if
you just plotted it you got a blob. Using statistical clustering
approaches we limited
the display to just the anomolies (those tracks that deviated from
normal shipping
lanes).
The technical tools are out there to support this (Spark, WPS)
but you would need
define the goals of your visualization.
On 5/31/17 12:53 PM, Michael Bowen
wrote:
Hello All,
Been playing with Geomesa for the last month or so. Primary
use case is to perform visualizations of a large amount of
geospatial sensor data (anywhere from a few hundred gigabytes
to a few terabytes). Currently, I'm ingesting the data into
Cassandra data store as a simple feature type (lat, lon pairs
with sensor readings over time).
When I don't use geowebcache I can display a few hundred
thousand readings from the cassandra data store and
dynamically interact with it in the browser. When using
geowebcache I can scale to a few million points. This is all
on a single node, with a cassandra replication factor of 2.
I would like to be able to display possibly billions of
points of data. My overarching question is about scalability
and performance - I'm fairly new to displaying geospatial data
on this scale, and am wondering what routes I should try and
explore next in geomesa to visualize the vast stores of data.
My initial thought is to query the data into a geomesa spark
spatial rdd, aggregate the readings into spatial bins, put the
data into a separate data store, and then display the
aggregated data results. Any tips/advice is greatly
appreciated!
Cheers,
Mike
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users
--
========= mailto:dboyd@xxxxxxxxxxxxxxxxx ============
David W. Boyd
VP, Data Solutions
10432 Balls Ford, Suite 240
Manassas, VA 20109
office: +1-703-552-2862
cell: +1-703-402-7908
============== http://www.incadencecorp.com/ ============
ISO/IEC JTC1 WG9, editor ISO/IEC 20547 Big Data Reference Architecture
Chair ANSI/INCITS TC Big Data
Co-chair NIST Big Data Public Working Group Reference Architecture
First Robotic Mentor - FRC, FTC - www.iliterobotics.org
Board Member- USSTEM Foundation - www.usstem.org
The information contained in this message may be privileged
and/or confidential and protected from disclosure.
If the reader of this message is not the intended recipient
or an employee or agent responsible for delivering this message
to the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication
is strictly prohibited. If you have received this communication
in error, please notify the sender immediately by replying to
this message and deleting the material from any computer.