Re: [geomesa-users] Adaption RowId, Spatial Bin Level

[Chris Eichelberger <cne1x@xxxxxxxx>]

Marcel,

One other note that came up from internal discussions about your
question:  Customizing the index-schema format is something that we
recommend most people do *not* do, because:

1.  this function is deprecated;

2.  it is very easy to introduce errors;

3.  it requires significant background knowledge about your data and
your cluster in order to tune the value properly;

4.  unless you have very specialized data, it is probably unnecessary

Sorry for any confusion!

Sincerely,
  -- Chris

On Mon, 2015-09-14 at 12:45 -0400, Chris Eichelberger wrote:
> Marcel,
> 
> There is some in-source documentation on how to construct this string
> here:
> 
> 
> https://github.com/locationtech/geomesa/blob/master/geomesa-accumulo/geomesa-accumulo-datastore/src/main/scala/org/locationtech/geomesa/accumulo/index/IndexSchema.scala#L21-L55
> 
> An easier (and better) way is to use the IndexSchemaBuilder class later
> in the same file:
> 
> 
> https://github.com/locationtech/geomesa/blob/master/geomesa-accumulo/geomesa-accumulo-datastore/src/main/scala/org/locationtech/geomesa/accumulo/index/IndexSchema.scala#L330
> 
> There is an example of how to use this in the unit test:
> 
> 
> https://github.com/locationtech/geomesa/blob/master/geomesa-accumulo/geomesa-accumulo-datastore/src/test/scala/org/locationtech/geomesa/accumulo/index/IndexSchemaTest.scala#L174-L183
> 
> Sincerely,
>   -- Chris
> 
> 
> On Mon, 2015-09-14 at 18:13 +0200, Marcel wrote:
> > I found the simplified version for the default index-schema format with 
> > 10 bits:  YXTTYXTTYX
> > How does the default index-schema format exactly looks like?
> > 
> > featureType.getUserData().put(Constants.SFT_INDEX_SCHEMA, "?");
> > 
> > Thanks,
> > Marcel Jacob.
> > 
> > 
> > Am 14.09.2015 14:56, schrieb Chris Eichelberger:
> > > Marcel,
> > >
> > > Note:  The following discussion, while somewhat interesting, is also
> > > deprecated:  The index-schema format described here was specific to the
> > > "_st_idx" table that is being replaced by the "_z3" table and index
> > > structure that we have found often out-performs the older index.
> > >
> > > Deprecated "_st_idx" index-schema format...  Table III's "spatial bins"
> > > column refers to the number of (base-32-encoded) Geohash characters that
> > > appear in the RowID.  Our experience was, generally, that the best
> > > performance occurred when the specificity of the RowID was appropriate
> > > for the granularity of data being indexed.  That is to say, if you have
> > > global data that is nearly uniformly distributed, a single (5-bit,
> > > base-32) Geohash character in the RowID may be appropriate; but for data
> > > concerning a single metropolitan area, 3-4 Geohash characters in the
> > > RowID might provide for better differentiation among rows.  This setting
> > > was controlled by what we called the "index-schema format", and while
> > > there is a setting to do this, it is not widely publicized.  (You have
> > > to set a user-data value for the "geomesa.index.st.schema" key within
> > > the simple-feature type when the schema is first registered in GeoMesa.)
> > >
> > > I hope this helps to address your interest.  If you have additional
> > > questions, please continue to share them.
> > >
> > > Thanks!
> > >
> > > Sincerely,
> > >    -- Chris
> > >
> > >
> > > On Mon, 2015-09-14 at 14:22 +0200, Marcel wrote:
> > >> Hello all,
> > >> I have found in your paper on page 7 (Table III) that increasing the
> > >> number of spatial bin level (maybe from 1 to 2) can have a positive
> > >> effect on query response times. Can you tell me, where I can set this
> > >> adaption of the rowId for my mapreduce job? Would you recommend a
> > >> spatial bin level of 2 or even higher?
> > >>
> > >> Thanks,
> > >> Marcel Jacob.
> > >> _______________________________________________
> > >> geomesa-users mailing list
> > >> geomesa-users@xxxxxxxxxxxxxxxx
> > >> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> > >> http://www.locationtech.org/mailman/listinfo/geomesa-users
> > >
> > > _______________________________________________
> > > geomesa-users mailing list
> > > geomesa-users@xxxxxxxxxxxxxxxx
> > > To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> > > http://www.locationtech.org/mailman/listinfo/geomesa-users
> > 
> > _______________________________________________
> > geomesa-users mailing list
> > geomesa-users@xxxxxxxxxxxxxxxx
> > To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> > http://www.locationtech.org/mailman/listinfo/geomesa-users
> 
> 
> _______________________________________________
> geomesa-users mailing list
> geomesa-users@xxxxxxxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> http://www.locationtech.org/mailman/listinfo/geomesa-users