Thanks Emilio for quick
response. Below are the required
details. Regarding update from
2.3.2 to 2.4.0, we can upgrade
it but it will require some
effort and time which I want to
avoid for now.
Exact filter
which I am using: BBOX(geometry,-180.0,-90.0,180.0,90.0) AND ingestionTimestamp <= '2019-12-23 06:00:00' AND nextTimestamp > '2019-12-23 06:00:00'
hbase(main):002:0> scan 'atlas'
ROW COLUMN+CELL
OSMNodes~attributes column=m:v, timestamp=1577234629875, value=*geometry:Point:srid=4326,ingestionTimestamp:Timestamp,nextTimestamp:Timestamp,serializerVersion:String,featurePayload:String;geomesa.index.dtg='ingestionTimestamp',geomesa.z.splits='60',geomesa.indices='z3:6:3:geometry:ingestionTimestamp,id:4:3:'
OSMNodes~stats-date column=m:v, timestamp=1577234629875, value=2019-12-25T00:43:49.836Z
OSMNodes~table.id.v4 column=m:v, timestamp=1577234646266, value=atlas_OSMNodes_id_v4
OSMNodes~table.z3.geometry.ingestionTimestamp.v6 column=m:v, timestamp=1577234629897, value=atlas_OSMNodes_z3_geometry_ingestionTimestamp_v6
OSMRelationMembers~attributes column=m:v, timestamp=1577234747359, value=ingestionTimestamp:Timestamp,relationId:String,featureTypeId:String,serializerVersion:String,featurePayload:String;geomesa.index.dtg='ingestionTimestamp',geomesa.indices='attr:8:3:relationId:ingestionTimestamp,attr:8:3:featureTypeId:ingestionTimestamp,id:4:3:'
OSMRelationMembers~stats-date column=m:v, timestamp=1577234747359, value=2019-12-25T00:45:47.320Z
OSMRelationMembers~table.attr.featureTypeId.ingestionTimestamp.v8 column=m:v, timestamp=1577234751575, value=atlas_OSMRelationMembers_attr_featureTypeId_ingestionTimestamp_v8
OSMRelationMembers~table.attr.relationId.ingestionTimestamp.v8 column=m:v, timestamp=1577234747380, value=atlas_OSMRelationMembers_attr_relationId_ingestionTimestamp_v8
OSMRelationMembers~table.id.v4 column=m:v, timestamp=1577234755743, value=atlas_OSMRelationMembers_id_v4
OSMRelations~attributes column=m:v, timestamp=1577234692949, value=*geometry:MultiPolygon:srid=4326,ingestionTimestamp:Timestamp,nextTimestamp:Timestamp,serializerVersion:String,featurePayload:String;geomesa.index.dtg='ingestionTimestamp',geomesa.z.splits='60',geomesa.indices='xz3:2:3:geometry:ingestionTimestamp,id:4:3:'
OSMRelations~stats-date column=m:v, timestamp=1577234692949, value=2019-12-25T00:44:52.909Z
OSMRelations~table.id.v4 column=m:v, timestamp=1577234710295, value=atlas_OSMRelations_id_v4
OSMRelations~table.xz3.geometry.ingestionTimestamp.v2 column=m:v, timestamp=1577234692970, value=atlas_OSMRelations_xz3_geometry_ingestionTimestamp_v2
OSMTestNodes~attributes column=m:v, timestamp=1577143864743, value=*geometry:Point:srid=4326,ingestionTimestamp:Timestamp,nextTimestamp:Timestamp,serializerVersion:String,featurePayload:String;geomesa.index.dtg='ingestionTimestamp',geomesa.z.splits='60',geomesa.indices='z3:6:3:geometry:ingestionTimestamp,id:4:3:'
OSMTestNodes~stats-date column=m:v, timestamp=1577143864743, value=2019-12-23T23:30:56.200Z
OSMTestNodes~table.id.v4 column=m:v, timestamp=1577143890005, value=atlas_OSMTestNodes_id_v4
OSMTestNodes~table.z3.geometry.ingestionTimestamp.v6 column=m:v, timestamp=1577143864809, value=atlas_OSMTestNodes_z3_geometry_ingestionTimestamp_v6
OSMWayNodes~attributes column=m:v, timestamp=1577234724952, value=ingestionTimestamp:Timestamp,wayId:String,nodeId:String,serializerVersion:String,featurePayload:String;geomesa.index.dtg='ingestionTimestamp',geomesa.indices='attr:8:3:wayId:ingestionTimestamp,attr:8:3:nodeId:ingestionTimestamp,id:4:3:'
OSMWayNodes~stats-date column=m:v, timestamp=1577234724952, value=2019-12-25T00:45:24.908Z
OSMWayNodes~table.attr.nodeId.ingestionTimestamp.v8 column=m:v, timestamp=1577234729162, value=atlas_OSMWayNodes_attr_nodeId_ingestionTimestamp_v8
OSMWayNodes~table.attr.wayId.ingestionTimestamp.v8 column=m:v, timestamp=1577234724973, value=atlas_OSMWayNodes_attr_wayId_ingestionTimestamp_v8
OSMWayNodes~table.id.v4 column=m:v, timestamp=1577234733300, value=atlas_OSMWayNodes_id_v4
OSMWays~attributes column=m:v, timestamp=1577234660315, value=*geometry:LineString:srid=4326,ingestionTimestamp:Timestamp,nextTimestamp:Timestamp,serializerVersion:String,featurePayload:String;geomesa.index.dtg='ingestionTimestamp',geomesa.z.splits='60',geomesa.indices='xz3:2:3:geometry:ingestionTimestamp,id:4:3:'
OSMWays~stats-date column=m:v, timestamp=1577234660315, value=2019-12-25T00:44:20.278Z
OSMWays~table.id.v4 column=m:v, timestamp=1577234677610, value=atlas_OSMWays_id_v4
OSMWays~table.xz3.geometry.ingestionTimestamp.v2 column=m:v, timestamp=1577234660337, value=atlas_OSMWays_xz3_geometry_ingestionTimestamp_v2
Hello,
If possible, can you upgrade
to 2.3.2 or 2.4.0? There is
at least one bug that may
cause that behavior, that
has been fixed in those
versions. Aside from that,
it would be helpful if you
could provide the exact
filters you're using, and
the data from your catalog
table in hbase, as output by
scanning it through the
hbase shell.
Thanks,
Emilio
On 1/15/20 2:58 PM,
Amit Srivastava wrote:
Hi All,
I am using Geomesa
v2.3.0 with HBase.
While running BBOX
export query on
Geomesa the results
are inconsistent. I am
seeing few features
are missing in export.
Can someone help in
debugging this?
What
investigation I have
done so far?
- Compare the data
ingested vs
exported. Found
below diff for it.
- Checked the
Application logs
which is calling
Geomesa via GeoTools
interface. I am not
seeing any error in
the application log.
- Checked HBase
related metrics and
logs. I am not
seeing any error
during the time when
we performed the
execution.
- Re-Run the query
on a smaller BBOX
which has missing
data, the reported
missing data in 2nd
export got returned
by Geomesa in 3rd
query. Hence I can't
find repreoducable
steps for this
issue.
Missing Features
In 2nd Export Run:
Missing Features
in 1st Export Run:
--
Regards,
Amit Kumar
Srivastava
--
Regards,
Amit Kumar Srivastava