| Re: [geomesa-users] Load and query entire Gdelt Dataset |
Raffaele,
If the gdelt_records table is empty (as well as the gdelt_attr_idx and
gdelt_st_idx tables), then yes, I suspect that ingest failed.
The first suspect I would check is whether the data format in the file
you downloaded exactly matches the FeatureType specification you
expected. Otherwise, the records will all be processed, but discarded,
during ingest. Not only do the types have to align, but (if I remember
this rightly) the columns must be specified in order, and none can be
omitted. If the input file contains 67 columns, then the feature type
specification should also contain 67 elements whose types match.
Sincerely,
-- Chris
On Tue, 2015-02-24 at 22:53 +0100, Raffaele Palmieri wrote:
> Hi Chris, looking at Accumulo Monitor, it shows me gdelt table with 10
> entries and many gdelt_records tables, all with 0 entries. Is it
> possible that ingest job failed?
>
> I tried with GDELT.MASTERREDUCEDV2.TXT file, downloadable from Gdelt
> archive.
>
> Sincerely,
>
> Raffaele,
>
>
> 2015-02-24 19:44 GMT+01:00 Chris Eichelberger <cne1x@xxxxxxxx>:
> Raffaele,
>
> Is TIME enabled as a dimension on your layer in GeoServer? If
> so, you
> probably need to provide a TIME clause as part of the WMS
> request.
>
> Does the Accumulo web monitor show there being records in the
> various
> tables for this feature?
>
> Is your WMS query for the entire world and all time, or is it
> constrained to a smaller geography and time?
>
> Thanks!
>
> Sincerely,
> -- Chris
>
>
> On Tue, 2015-02-24 at 19:42 +0100, Raffaele Palmieri wrote:
> > Hi to all, I am trying to load entire GDelt Dataset,
> >
> > the job seems to work fine, following is log:
> > 15/02/24 19:13:22 INFO mapreduce.Job: Counters: 44
> > File System Counters
> > FILE: Number of bytes read=6
> > FILE: Number of bytes written=2098146
> > FILE: Number of read operations=0
> > FILE: Number of large read operations=0
> > FILE: Number of write operations=0
> > HDFS: Number of bytes read=6584558010
> > HDFS: Number of bytes written=0
> > HDFS: Number of read operations=76
> > HDFS: Number of large read operations=0
> > HDFS: Number of write operations=0
> > Job Counters
> > Killed map tasks=3
> > Launched map tasks=28
> > Launched reduce tasks=1
> > Data-local map tasks=28
> > Total time spent by all maps in occupied
> slots
> > (ms)=3672817
> > Total time spent by all reduces in occupied
> slots
> > (ms)=540096
> > Map-Reduce Framework
> > Map input records=87298047
> > Map output records=0
> > Map output bytes=0
> > Map output materialized bytes=150
> > Input split bytes=2875
> > Combine input records=0
> > Combine output records=0
> > Reduce input groups=0
> > Reduce shuffle bytes=150
> > Reduce input records=0
> > Reduce output records=0
> > Spilled Records=0
> > Shuffled Maps =25
> > Failed Shuffles=0
> > Merged Map outputs=25
> > GC time elapsed (ms)=92419
> > CPU time spent (ms)=463190
> > Physical memory (bytes) snapshot=7457234944
> > Virtual memory (bytes) snapshot=22437404672
> > Total committed heap usage
> (bytes)=5193990144
> > Shuffle Errors
> > BAD_ID=0
> > CONNECTION=0
> > IO_ERROR=0
> > WRONG_LENGTH=0
> > WRONG_MAP=0
> > WRONG_REDUCE=0
> > File Input Format Counters
> > Bytes Read=6584555135
> > File Output Format Counters
> > Bytes Written=0
> >
> >
> > but from geoserver with wms request I don't see red squares
> on a white
> > tile.
> >
> > Some suggestions to understand if data are loaded correctly?
> >
> > Thanks in advance,
> >
> > Raffaele.
> >
> >
>
> > _______________________________________________
> > geomesa-users mailing list
> > geomesa-users@xxxxxxxxxxxxxxxx
> > To change your delivery options, retrieve your password, or
> unsubscribe from this list, visit
> > http://www.locationtech.org/mailman/listinfo/geomesa-users
>
>
> _______________________________________________
> geomesa-users mailing list
> geomesa-users@xxxxxxxxxxxxxxxx
> To change your delivery options, retrieve your password, or
> unsubscribe from this list, visit
> http://www.locationtech.org/mailman/listinfo/geomesa-users
>
>
> _______________________________________________
> geomesa-users mailing list
> geomesa-users@xxxxxxxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> http://www.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users