Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] Query over different geometry types?
It should work the same... your code looks fine - just make sure you call 'close' on the feature iterator once you're done with it. How long did you let it hang for? Possibly it is still running scans against HBase, but they just aren't returning any rows.

Do you need to use GeoMesa 1.3.1 for any reason? We've made several bug fixes related to HBase scans since then, upgrading to 1.3.4 may fix your problem (although we haven't seen this particular issue before).

thanks,

Emilio

On 11/21/2017 07:17 PM, Jason Koh wrote:
Sure I will report any results once we have.


Before then, I faced an irregular behavior after the change. The query result iterator's "hasNext()" does not return.

The test configuration is
A Ubuntu 16.04 VM with 3 cores and 6GB RAM.
HBase 1.2.6 standalone and Geomesa. 1.3.1

I was iterating the query result like this.

```java
FeatureIterature iter = featureSource.getFeatures(query).features();
while (iter.hasNext() {
  Feature feat = iter.next();
  // doing something with feat.
}
```
I observed that it properly returned all the data for the query, and then it's held at hasNext(). The dataset just has a two Points and the query BBOX is fairly small. It runs properly if I use "Point" as a default geometry type, but stops with "Geometry" as a default geometry type.

Was there something that I should've specified further?

Thanks!


With regards,
Jason Koh

On Tue, Nov 21, 2017 at 1:29 PM, Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx> wrote:
Let us know how it goes - I don't think we have a lot of data on how the precision of the index affects queries, so any insight you can provide would be helpful.

thanks,

Emilio


On 11/21/2017 02:16 PM, Jason Koh wrote:
Hi Emilio,

Thanks for all the clarifications. My datasets are usually with small geometry sizes for now. I will play with the configurations as you suggested.

Thank you again.


With regards,
Jason Koh

On Tue, Nov 21, 2017 at 11:09 AM, Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx> wrote:
Hi Jason,

Just to clarify, I believe that the method 'setDefaultGeometry' on a simple feature is a short-cut for setting the attribute value, not which attribute is the default geometry. I.e. in your case, feature.setDefaultGeometry(geom) is equivalent to feature.setAttribute("loc", geom).

GeoMesa uses an XZ index for non-point geometry types. You can control the precision of this index when creating your SimpleFeatureType, which may give you better performance for points, at the expense of worse performance for geometries with larger bounding boxes. You should try playing around with different values with your data set and typical query patterns, and see if it makes a difference. Details are here: http://www.geomesa.org/documentation/user/datastores/index_config.html#configuring-xz-index-precision

I don't have any hard numbers on performance differences between point and non-point indices. Generally, non-point data sets tend to be smaller, so performance is still good overall.

Thanks,

Emilio

On 11/21/2017 01:57 PM, Jason Koh wrote:
Hi Jim,

Thanks a ton for the useful comments.

I would like to clarify if my understanding is correct also. Here's what I'd do with your comments.

```java
// Define Feature Type
SimpleFeatureTypeBuilder b = new SimpleFeatureTypeBuilder();
b.setName("TestFeatureType");
b.add("loc", Geometry.class, 4326);
b.add("date", Date.class);
b.add("value", String.class);
b.setDefaultGeometry("loc");
SimpleFeatureType sft = b.buildFeatureType();
sft.getUserData().put("geomesa.mixed.geometries", "true");

dataStore.createSchema(sft); // Assume dataStore is loaded.

// Write a feature
FeatureWriter writer = dataStore.getFeatureWriterAppend("TestFeatureType");

// Adding a sensor value with geometry type "LineString"
SimpleFeature newFeature = writer.next();
newFeature.setAttribute("value", "TestValue1");
newFeature.setAttribute("date", SomeDate);
newFeature.setAttribute("loc", line); // line is given.
```
Is this correct? It returned correct results from the query, but a query for Points got significantly slowed. It's reasonable but do you have any numbers about how much performance will be degraded by using "Geometry" as a default index instead of "Point"?


By the way, I forgot to put below code in the previous email:
```java
// feat is an instance of SimpleFeature, not SimpleFeatureType
neaFeature.setDefaultGeometry("line_loc");
```
There is a method in SimpleFeature that sets default geometry type, so I was curious if it actually does set proper index per feature instance. My understanding of your comments is this is not working.

Thank you very much.



With regards,
Jason Koh

On Mon, Nov 20, 2017 at 6:16 PM, Jim Hughes <jnh5y@xxxxxxxx> wrote:
Hi Jason,

Great questions.  I'll re-paste some of the sections and respond inline.  The answer to your first question is 'yes'.:)

1. Is it okay to define multiple geometry types into a single feature type?

Yes.  The chief thing to observe is that the default geometry field is the one used for indexing in GeoMesa. 

2. Can default geometry type be changed per every feature?

No.  The default geometry field is managed per SimpleFeatureType.  (This is part of the GeoTools api.)

3. How can I properly get query results for the above case?
4. Is there anyway that I can query both point_loc and line_loc with a single query? Should I just need to OR two geometry filters? (Is there a better way?)

 As an important piece of information, GeoMesa can store and index a geometry column of type 'Geometry' (meaning that there could be a mix of points, lines, etc).  Since GeoMesa optimizes for Point and non-Point geometry types differently, one does have to give a hint to let the system know that being general wasn't a mistake.  See http://www.geomesa.org/documentation/user/datastores/index_config.html#mixed-geometry-types for the hint which needs to be set during SFT creation.

From what you've said, it sounds like your data sometimes has a point or a line, and at ingest time, you could say which field should be used for general purpose querying.  I'd suggest that you can either have an additional mixed geometry field or you can replace one or both of your current geometry fields with a mixed one.

The mixed geometry column would be the default geometry field (and hence indexed).  Your query simplifies to an query on that one field.

If that solution doesn't work, do you have any real-world constraints/relationships between the point_loc and line_loc which could be leveraged?  As an example, suppose that the point_loc was always on the line_loc and that line measured less than 0.1 degrees (around 11km), then one could use an index on point_loc by buffering the bbox by the upper bound for the line length. 

I'm suggesting that one avoid using an OR across two geometry fields since that'll likely result in a full-table scan. 

Cheers,

Jim

On 11/20/2017 8:18 PM, Jason Koh wrote:
Hi,

I am using Geomesa as a database of various sensor measurements. Each sensor's geometry type may vary from Point and LineString to Polygon. Is it possible to store all of them as a feature type and query in a single way?

I currently implement each geometry type as different attributes and define default geometry per feature.

The simplified version of my implementation is:

### Adding an entry

```java
// Define Feature Type
SimpleFeatureTypeBuilder b = new SimpleFeatureTypeBuilder();
b.setName("TestFeatureType");
b.add("point_loc", Point.class, 4326);
b.add("line_loc", LineString.class, 4326)
b.add("date", Date.class);
b.add("value", String.class);
b.setDefaultGeometry("point_loc");
SimpleFeatureType featType = b.buildFeatureType();

dataStore.createSchema(featType); // Assume dataStore is loaded.

// Write a feature
FeatureWriter writer = dataStore.getFeatureWriterAppend("TestFeatureType");

// Adding a sensor value with geometry type "LineString"
SimpleFeature newFeature = writer.next();
newFeature.setAttribute("value", "TestValue1");
newFeature.setAttribute("date", SomeDate);
newFeature.setAttribute("line_loc", line); // line is given.
// Not adding point ad line is included.
```

### Querying the value.
```
String cqlGeometry = "BBOX( line_loc, x1, y1, x2, y2)" // Given BBOX includes the point added above.
Filter filter = CQL.toFilter(cqlGeometry);
Query q = new Query("TestFeatureType", filter);
q.getHints().put(QueryHints.LOOSE_BBOX(), Boolean.FALSE);
FeatureSource<SimpleFeatureType, SimpleFeature> featureSource = dataStore.getFeatureSource("TestFeatureType");
FeatureIterator<SimpleFeature> featureItr = featureSource.getFeatures(query).features();

// This iterator turns out to be empty.
while (featureItr.hasNext()) {
  Feature feat = featureItr.next();
  System.out.println(feat);
}
```

My questions are
1. Is it okay to define multiple geometry types into a single feature type?
2. Can default geometry type be changed per every feature?
3. How can I properly get query results for the above case?
4. Is there anyway that I can query both point_loc and line_loc with a single query? Should I just need to OR two geometry filters? (Is there a better way?)


Thank you so much and I hope that one day I can share the entire project as an instance of using Geomesa.



With regards,
Jason Koh


_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users





_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users


_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users



_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users


_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users



_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users

Back to the top