Hi again,
Sorry for the second message. I didn't read the title completely
before responding.
In general, Accumulo has some issues with high concurrency (see
http://ieee-hpec.org/2013/index_htm_files/28-2868615.pdf).
Are you writing data to GeoMesa frequently during your analysis? If
so, increasing
tserver.mutation.queue.max to 4m
(http://accumulo.apache.org/release_notes/1.5.1.html). Similarly,
check out the release notes for 1.5.2
(http://accumulo.apache.org/release_notes/1.5.2.html). Some of
those updates may help with your performance issues.
Since you mention using attribute filters, I'd suggest checking out
the AttributeIndexJob:
https://github.com/locationtech/geomesa/tree/accumulo1.5.x/1.x/geomesa-jobs.
Note that you will need to update the indexed attributes manually
after the job has finished.
Thanks,
Jim
On 03/02/2015 12:54 AM, Jim Hughes
wrote:
Hi,
In general, when using Storm, you will want to find ways to
distribute your computation so that you can use more than one
bolt. Can you say more about your spatial analysis?
In terms of the GeoMesa read performance, there are several
factors. Generally, GeoMesa will perform well with
spatio-temporal queries. If your query involves non-spatial,
non-temporal attributes, you may be able to increase performance
with our secondary attribute indexes. Some factors will be
related to the distribution of your data and your queries. Your
cloud's configuration and hardware will factor in as well. Can
you say something about your queries and the performance you are
seeing?
To see Accumulo's performance, you can visit the Accumulo monitor
page hosted on the Accumulo master on port 50095. As you make a
scan, you should see feedback on the number of entries scanned and
returned per second.
How are you running Kafka and Storm? If you are testing on one
machine, I wonder if you may be running into memory trouble.
If you can provide some more details, others on the list and I may
be able to help more.
Thank you for your question,
Jim
On 02/28/2015 08:24 AM, 焦小超 wrote:
Hi,
We use geomesa to
perform spatial analysis in storm + kafka application,
one kafka spout, one spatial analysis bolt. when we send
more data in less time to kafka, the latency of spatial
analysis bolt will higher and higher! In the func exec
of spatial analysis bolt, we invoke
geomesa query process as the following code :
.....
featureSource.getFeatures(query)
.....
We want to know which
factors involved in geomesa's read performance?
Thanks a lot!
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users
|