Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-dev] Fwd: Hbase spark-sql for filter `Map` `List`

hello , Emilio,
      Thank you for you detail description. I will attempt that . 

Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx> 于2019年1月12日周六 上午12:33写道:
Hello,

I don't think that it is currently possible to do that. Spark sql doesn't natively support list or map types. GeoMesa (not through spark) can query on list types, but due to our use of GeoTools CQL there isn't currently a way to query on map types. So it would be somewhat complicated to implement.

It may be more feasible to store your map as a json object string. Spark sql has functions for parsing and filtering json, so you could do something like:

select get_json_object(properties, '$.k1') from test_road_value where get_json_object(properties, '$.k2') = 'vvv'

Note that this would not be an optimized query, and would have to load your entire dataset from GeoMesa and then do the filtering in spark.

In your GeoMesa schema, you could just use a plain string or make it a json object. Making it a json object wouldn't affect spark sql, but may be useful if you are not using spark: https://www.geomesa.org/documentation/user/geojson.html#json-attributes

Thanks,

Emilio

On 1/11/19 2:40 AM, 段卫东 wrote:


---------- Forwarded message ---------
From: <geomesa-dev-owner@xxxxxxxxxxxxxxxx>
Date: 2019年1月11日周五 下午3:32
Subject: Hbase spark-sql for filter `Map` `List`
To: <misslzyan108@xxxxxxxxx>


You must subscribe to this list before posting.  Please go to
https://locationtech.org/mailman/listinfo to subscribe to the list,
then try again.  Perhaps you are already subscribed using another
email address; if that is the case, please send email to this list
using the email address you've used to subscribe.




---------- Forwarded message ----------
From: "段卫东" <misslzyan108@xxxxxxxxx>
To: geomesa-dev@xxxxxxxxxxxxxxxx
Cc: 
Bcc: 
Date: Fri, 11 Jan 2019 15:32:21 +0800
Subject: Hbase spark-sql for filter `Map` `List`
hi ,all,

       I want to use spark-sql to filter with `Map`  `List` data type .

      like this :
     
     select properties['k1'] 
       from test_road_value where properties['k2']='vvv'

     Did anyone  going to do  this ?  and/or have the same request ?

_______________________________________________
geomesa-dev mailing list
geomesa-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-dev

_______________________________________________
geomesa-dev mailing list
geomesa-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-dev

Back to the top