Ben,
Interesting question. Can you give more info on
what a failed ingest
looks like?
We've thought a lot about a write-ahead log based
approach to ingest for
performance reasons but I think it could apply in
your scenario as well.
It's basically the idea of the staging table.
Load data into the
staging table and flush it to the main table.
Depending on how
up-to-date you need for your queries, you'll need
to hit both the main
table and the staging table.
There may be some clever things you can do with
Accumulo's facility for
cloning tables to support this.
http://accumulo.apache.org/1.7/accumulo_user_manual#_cloning_tables
In terms of update, you can use a regular
FeatureWriter to update
records. It is much slower than the appending
FeatureWriter because it
needs to check the original record and delete any
index entries.
Thanks,
Anthony
Benjamin Weaver
<Benjamin.Weaver@xxxxxxxxxxxxxxxxxx> writes:
> Hi all,
>
>
> We are using Geomesa 1.2.1 on Accumulo 1.7.2.
We are seeking to implement a rollback procedure
to use during Ingest. This is our first priority;
a second would be the ability to update data
already in the database. We lack the space to
maintain a backup copy of our large tables. We
have one large Accumulo table containing 1 geomesa
SimpleFeature.
>
>
> Load of a staging table, followed by geomesa
cmd-line export of this table, and merge of this
export into our main table, shows some promise as
a hedge during ingest. Another approach would seem
represented by the Accumulo command line, which
enables row deletions but I did not know whether
Accumulo shell could gracefully handle the Geomesa
table suite.
>
>
> But what is the best way to rollback on
mid-way failure of ingest? Are examples available?
I saw some Scala classes that include calls to
trans.rollback(), etc. Are these or other classes
to be used for batch, or bulk or total rollback?
Or are there other rollback tools and techniques I
have overlooked?
>
>
> Any perspectives are appreciated and welcome!
>
>
> Ben Weaver
>
> This email (and any attachments) may contain
confidential information and is intended solely
for the recipient(s) to whom the email is
addressed. If you received this email in error,
please inform us immediately and delete the email
and all attachments without further using, copying
or disclosing the information. This email and any
attachments are believed to be, but cannot be
guaranteed to be, secure or virus-free. Satellite
Applications Catapult Limited is registered in
England & Wales. Company Number: 7964746.
Registered office: Electron Building, Fermi
Avenue, Harwell Oxford, Didcot, Oxfordshire OX11
0QR.
>
_______________________________________________
> geomesa-users mailing list
>
geomesa-users@xxxxxxxxxxxxxxxx
> To change your delivery options, retrieve
your password, or unsubscribe from this list,
visit
>
https://www.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your
password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
