[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
Re: [geomesa-users] Question about min and max times in indexing
|
Yeah, that error is a bit obtuse but it's coming from converting the
date into an index value. I believe that currently if a feature fails to
validate for any index, it will not be stored at all. This is to prevent
partial indexing, where your query results might differ based on which
index it uses. Previously we allowed partial indexing, and I think at
this point we'd like to support both based on a configuration property,
but haven't implemented it yet.
We haven't really had any use-cases so far for storing data that old, so
we don't currently support it. However, there are a couple things you
could do (off the top of my head):
* Add another date field for indexing, or disable the z3 index. If the
date isn't part of the primary z index, then it won't cause any
problems. You can still filter on it as normal, it just won't use the
date in the primary range planning so queries will be slower. To
alleviate that, you could add an attribute index on the date field -
that does not have the same restrictions on date range, but it is not a
composite index so query planning will use either date *or* geometry but
not both.
* Offset dates by some fixed amount to bring them into an indexable
range, and add some logic in your client to transform queries and
results. This may be fairly complicated...
From a technical perspective I don't think there is any reason we
couldn't store dates before the epoch, it just hasn't been implemented.
Thanks,
Emilio
On 04/20/2017 10:13 PM, David Boyd wrote:
Emilio:
Thanks. I puzzled it out in the end.
How would one date index historical data? The data I have has
numerous dates before the Epoch. The exception I am
getting below. Does this mean my feature did not get stored, or just
the date was not indexed? If the latter, how would
this data behave on a query including the date?
2017-04-20 17:11:12,306 | WARN | [Thread-7] |
(ICEWS_EntityExtractor.java:240) - StartDateString: 1968-01-01
StartDate: 1968-01-01T00:00:00.000-05:00 EndDateString: 1996-08-31
EndDate: 1996-08-31T00:00:00.000-04:00
2017-04-20 17:11:12,306 | INFO | [Thread-7] |
(ICEWS_EntityExtractor.java:300) - Persisting 2 ICEWS records.
2017-04-20 17:11:12,556 | ERROR | [Thread-7] |
(AccumuloPersistor.java:1073) - requirement failed: Value out of
bounds ([0.0 604800.0]): -241200.0
java.lang.IllegalArgumentException: requirement failed: Value out of
bounds ([0.0 604800.0]): -241200.0
at scala.Predef$.require(Predef.scala:224)
at
org.locationtech.geomesa.curve.NormalizedDimension$class.normalize(NormalizedDimension.scala:17)
On 4/20/17 6:07 PM, Emilio Lahr-Vivaz wrote:
Hi David,
I don't believe that this is in our documentation, but it's commented
in our source code. The min date will always be the unix epoch, and
the max date depends on the indexing interval of your z-curve (the
default interval is week):
https://github.com/locationtech/geomesa/blob/master/geomesa-z3/src/main/scala/org/locationtech/geomesa/curve/BinnedTime.scala#L15-L39
Thanks,
Emilio
On 04/20/2017 04:45 PM, David Boyd wrote:
All:
Haven't found this in the documents yet so I thought I would ask.
I have a two fields in my data representing a startTime and an endTime.
Values for those string fields are normally dates but can also be
"beginning of time" and
"end of time" respectively.
I originally I tried setting beginning of time to be 01/01/1111 but
I would get an
index out of range error (I assume it is because this was before the
standard Unix epoc).
That error was down in the XZ3 index creation.
I then tried using new DateTime(Long.MIN) and new DateTime(Long.MAX)
but the max
now throws errors in Joda.Time.
So what are the min and max Times supported by Geomesa in the indexes?
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users